Setting extra Python3 strategy variables in the Cmake call didn't solve the problem. They may have been necessary, but they weren't sufficient.
In the end, I installed Python 3.8.2, and pointed everything at that. For reasons I haven't discovered, Cmake seems to choose that one over the incomplete Python 3.7 distribution that's bundled with Visual Studio. (The obvious guess is that it's a higher version number, but the Cmake documentation denies that.) "Listen, strange women lying in ponds *Cmake* distributing swords *selecting Python installations* is no basis for a system of government *cross-platform build configuration.*" On Mon, May 11, 2020 at 4:34 PM Adrian McCarthy <amcca...@google.com> wrote: > Ug. After a rebase, this problem has again resurfaced for me. > > * PATH has only C:\Python36 > * cmake version 3.15.19101501-MSVC_2 > * cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON > -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1 > -DPYTHON_HOME=C:\Python36 > -DLLDB_PYTHON_HOME=C:\Python36 -DPython3_ROOT_DIR=C:\Python36 > -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja_dbg\bin\clang.exe > ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF > -DLLVM_ENABLE_PROJECTS="clang;lld;lldb;debuginfo-tests" > > Yet when linking LLDB, it goes looking at the Python 3.7 installation that > comes with Visual Studio. That wouldn't be much of a problem unless you're > trying to build debug, which I am. The VS version, doesn't come with debug > versions of the interpreter libraries, so the link fails: > > [1/7] Linking CXX shared library bin\liblldb.dll > FAILED: bin/liblldb.dll lib/liblldb.lib > cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual > Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" > -E vs_link_dll --intdir=tools\lldb\source\API\CMakeFiles\liblldb.dir > --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe > --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests -- > C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe > /nologo @CMakeFiles\liblldb.rsp /out:bin\liblldb.dll > /implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0 > /machine:x64 /debug /INCREMENTAL && cd ." > LINK Pass 1: command > "C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1424~1.283\bin\Hostx64\x64\link.exe > /nologo @CMakeFiles\liblldb.rsp /out:bin\liblldb.dll > /implib:lib\liblldb.lib /pdb:bin\liblldb.pdb /dll /version:11.0 > /machine:x64 /debug /INCREMENTAL /MANIFEST > /MANIFESTFILE:tools\lldb\source\API\CMakeFiles\liblldb.dir/intermediate.manifest > tools\lldb\source\API\CMakeFiles\liblldb.dir/manifest.res" failed (exit > code 1104) with the following output: > LINK : fatal error LNK1104: cannot open file 'python37_d.lib' > ninja: build stopped: subcommand failed. > > It looks like Cmake has added several more hints for finding Python > <https://cmake.org/cmake/help/v3.15/module/FindPython3.html#hints>. I'm > trying now with -DPython3_FIND_REGISTRY=LAST. > > I miss hermetic builds. > > On Thu, Feb 27, 2020 at 11:33 AM Adrian McCarthy <amcca...@google.com> > wrote: > >> > This was all fixed in CMake 3.12. >> >> For some definitions of "all fixed." ;-) >> >> It seems weird to me that FindPython3 found the VS-distributed Python, >> which isn't mentioned anywhere in the environment block, and that it chose >> that one over the Python 3 installation that was in the path and that >> included the interpreters and all of the corresponding libraries. >> >> > With FindPython{2,3} you know you'll have a matching interpreter and >> library. >> >> The build failure was that the Python distributed in VS does not have a >> debug version of the library (python37_d.lib), so you don't actually get a >> matching interpreter and library for debug builds. >> >> It's also not clear whether LLVM was consistently using the Python found >> the old way. My generated build.ninja file seemed to be using both >> versions inconsistently to run lit tests and the like. >> >> Anyway, thanks for the explanations. I've LGTMed your patch, which >> should eliminate future surprises when something else smuggles yet another >> version of Python onto a machine. >> >> On Thu, Feb 27, 2020 at 11:14 AM Jonas Devlieghere <jo...@devlieghere.com> >> wrote: >> >>> So to make my previous explanation more concrete: >>> >>> On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere < >>> jo...@devlieghere.com> wrote: >>> >>>> >>>> >>>> On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy <amcca...@google.com> >>>> wrote: >>>> >>>>> Thanks for the info. Setting Python3_ROOT_DIR solves the problem. >>>>> >>>>> Looking at the cmake output from before setting Python3_ROOT_DIR, >>>>> cmake looks for Python twice and finds it at the two different locations. >>>>> >>>>> Early on: >>>>> >>>>> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8") >>>>> >>>> >>> ^ This is using the "old" (CMake < 3.12) way of finding the Python >>> interpreter. >>> >>> >>>> >>>>> Which looks good (modulo the incorrect slash direction). But later: >>>>> >>>>> -- Found Python3: C:/Program Files (x86)/Microsoft Visual >>>>> Studio/Shared/Python37_64/python.exe (found version "3.7.5") found >>>>> components: Interpreter Development >>>>> -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual >>>>> Studio/Shared/Python37_64/libs/python37.lib >>>>> >>>> >>> ^ This is using the "new" (CMake > 3.12) way of finding the Python >>> interpreter and libraries. >>> >>> >>>> >>>>> Which is where the discrepancy comes in. Note that only C:\Python36 >>>>> is in my PATH. >>>>> >>>>> It's frustrating that this keeps breaking. Last time, I had to purge >>>>> all but one Python installation from my machine to get it to make a >>>>> consistent choice. But I just upgraded to VS 2019, and it smuggled in its >>>>> own version. >>>>> >>>>> So why are there two searches anyway? And why do they have different >>>>> algorithms that lead to different results? (I'm not sure _how_ it ever >>>>> found the Microsoft copy, since there's nothing in the process environment >>>>> that points that way.) >>>>> >>>> >>>> The reason there's two searches is because LLVM and LLDB have different >>>> requirements. LLVM just needs a python interpreter to run some scripts. >>>> LLDB on the other hand needs an interpreter and a matching Python library >>>> to link against. Before CMake 3.12, finding the interpreter and the >>>> libraries are two separate calls to find with no guarantees that they >>>> match. This lead to all kinds of issues, where you're linking against one >>>> version of Python and then trying to run the test suite with a totally >>>> different interpreter. There were other problems on Windows, which meant >>>> that we had our own hand-rolled implementation to find Python. >>>> >>>> This was all fixed in CMake 3.12. With FindPython{2,3} you know you'll >>>> have a matching interpreter and library. It also fixed all the problems we >>>> had to work around for Windows. Unfortunately, LLVM's minimum CMake version >>>> is 3.4, so we can't use it yet. For LLDB on Windows we agreed that the >>>> benefits of using FindPython3 are worth bumping the minimum required CMake >>>> version (see lldb/CMakeLists.txt, line 2-4). Once LLVM moves to CMake 3.12 >>>> or later, all these problems should be fixed. We can then call FindPython3 >>>> once and rely on everything being consistent. >>>> >>>> >>>>> >>>>> On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere < >>>>> jo...@devlieghere.com> wrote: >>>>> >>>>>> Hey Adrian, >>>>>> >>>>>> Config.h gets generated by expanding the corresponding CMake >>>>>> variables. If you look at LLDBConfig.cmake, you can see that >>>>>> LLDB_PYTHON_HOME is computed from PYTHON_EXECUTABLE. The problem appears >>>>>> that somehow CMake ignored your specified PYTHON_HOME and decided to >>>>>> pick a >>>>>> different Python. I'm not sure why though, because I use a similar CMake >>>>>> invocation on Windows. >>>>>> >>>>>> > cmake ..\llvm-project\llvm -G Ninja >>>>>> -DCMAKE_BUILD_TYPE=RelWithDebInfo >>>>>> -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF >>>>>> -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE >>>>>> -DPYTHON_HOME="C:/Program Files/Python36/" >>>>>> >>>>>> According to FindPython3 ( >>>>>> https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can >>>>>> set Python3_ROOT_DIR as a hint. Can you give that a try? If that works we >>>>>> should populate that variable from PYTHON_HOME in >>>>>> FindPythonInterpAndLibs.cmake. >>>>>> >>>>>> Cheers, >>>>>> Jonas >>>>>> >>>>>> On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev < >>>>>> lldb-dev@lists.llvm.org> wrote: >>>>>> >>>>>>> Is there documentation on how lldb\include\lldb\host\config.h is >>>>>>> generated? I'm again having the problem of the config trying to point >>>>>>> to >>>>>>> the wrong Python installation. >>>>>>> >>>>>>> When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like >>>>>>> this: >>>>>>> >>>>>>> cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON >>>>>>> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1 >>>>>>> -DPYTHON_HOME=C:\Python36 >>>>>>> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe >>>>>>> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF >>>>>>> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" >>>>>>> >>>>>>> But the generated Config.h contains: >>>>>>> >>>>>>> #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual >>>>>>> Studio/Shared/Python37_64" >>>>>>> >>>>>>> >>>>>>> And the mismatch causes my build to fail because it goes looking for >>>>>>> python37_d.dll, which is apparently not part of the Microsoft >>>>>>> distribution. >>>>>>> _______________________________________________ >>>>>>> lldb-dev mailing list >>>>>>> lldb-dev@lists.llvm.org >>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>>>> >>>>>> >>> I've put up a patch to use PYTHON_HOME as the hint: >>> https://reviews.llvm.org/D75275 >>> >>
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
