Hi, we are about to merge the wip-python-build-system. I'd like to add some commentary there explaining how the different install methods of Python behave. This could save someone else the effort of anaylsing and testing this again is some questions about the Python build system arise.
I ask you to review the text below. Since we want to merge soon, I'd ask you to review soon. Thanks. In Python there are different ways to install packages: distutils, setuptools, easy_install and pip. All of these are sharing the file setup.py, introduced with distutils in Python 2.0. setup.py can be considered as a kind of Makefile accepting targets (or commands) like "build" and "install". As of autumn 2016 the recommended way to install Python packages is using pip. For both distutils and setuptools running "python setup.py install" is the way to install Python packages. With distutils the "install" command basically copies all packages into <prefix>/lib/pythonX.Y/site-packages. Some time later "setuptools" have been established to enhance distutils. To use setuptools, the developer imports setuptools in setup.py. When importing setuptools, the original "install" command gets overwritten by setuptools' "install" command. easy_install and pip are both command-line tools capable to search and download the package source from PyPI (the Python Package Index). Both of them import setuptools and execute the "setup.py" file under their control. Thus the "setup.py" behaves as if the developer had imported setuptools within setup.py - even is still using only distutils. Setuptools' "install" command (to be more precise: the "easy_install" command which is called by "install") will put the path of the currently installed version of each package and it's dependencies (as declared in setup.py) into an "easy-install.pth" file. In guix each packages gets it's own "site-packages" directory and thus an "easy-install.pth" of it's own. To avoid conflicts this file gets renamed to <packagename>.pth in phase rename-pth-file. To ensure the .pth-file will be process, easy_install also creates a basic "site.py" in each "site-packages" directory. The file is the same for all packages, thus there is no need to rename it. The .pth-files contain the file-system paths (pointing to the store) of all dependencies. So the dependency is hidden in the .pth file but is not visible in the file-system. Now if packages A and B both required packages P, but in different versions, guix will not detect this when installing both A and B to a profile. (For details and example see https://lists.gnu.org/archive/html/guix-devel/2016-10/msg01233.html.) Now pip behaves a bit different: it always executes "setup.py" with the option "--single-version-externally-managed" set. This makes setuptools' "install" command to *not* run "easy_install" but the original "install" command and thus no .pth-file (and no site.py) will be created. The "site-packages" directory only contains the package and the related .egg-info directory. This is exactly what we need for guix and this is what we mimic in the install phase below. As a draw back, the magic of the .pth file of linking to the other required packages is gone and these packages have now to be declared as "propagated-inputs". Note: Importing setuptools also adds two sub-commands: "install_egg_info" and "install_scripts". These sub-commands are executed even if "--single-version-externally-managed" is set, thus the .egg-info directory and the scripts defined in entry-points will always be created. Note: Even if the "easy-install.pth" is not longer created, we kept this phase. There still may be packages creating an "easy-install.pth" manually for some good reason. -- Regards Hartmut Goebel | Hartmut Goebel | h.goe...@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |
