Hi, nex part of the analysis:
Result ======= The venv-hack I posted a few days ago works as expected only for GUIX_PROFILE, but not for virtual environments. Preliminary Proposal ======================= As it stands now, the venv-hack is not a valid solution. It may be the basis for another solution, tough. Rational =========== I tried answering four questions: A: Can virtual environments be stacked in stock Python? No, they can not, see point 1. and 2. below. C: Given PYTHONPATH is not set, do virtual environments in Guix use the correct "site" packages (which is the ones in GUIX_PROFILE). No, in guix venvs use the site-packages from /gnu/store/…-python-3.6.3/. See points 3. and 4 below. D: Would the venv-hack I posted a view days ago solve the issue? No, it would not. It would work as expected for python in the profile, but not for virtual environments based on this. See point 5. below. 1. Set up a virtual environemnt using the system installed python. ==================================================================== $ pyvenv-3.5 /tmp/venv-1 $ ls -l /tmp/venv-1/bin/python … /tmp/venv-1/bin/python -> python3.5 $ ls /tmp/venv-1/bin/ activate activate.fish easy_install-3.5* pip3* python@ python3.5@ activate.csh easy_install* pip* pip3.5* python3@ $ ls -l /tmp/venv-1/bin/python* … /tmp/venv-1/bin/python -> python3.5 … /tmp/venv-1/bin/python3 -> python3.5 … /tmp/venv-1/bin/python3.5 -> /usr/bin/python3.5 $ /tmp/venv-1/bin/python -m site sys.path = [ '/home/hartmut', '/usr/lib64/python35.zip', '/usr/lib64/python3.5', '/usr/lib64/python3.5/plat-linux', '/usr/lib64/python3.5/lib-dynload', '/tmp/venv-1/lib64/python3.5/site-packages', '/tmp/venv-1/lib/python3.5/site-packages', ] As expected there are only the venvs' site-packges in sys.path. 2. Now stack venv on top of venv-1. Use --system-site-packages to (hopefully) make venv-1's site-packages available to venv-2. ==================================================================== $ which pyvenv-3.5 /bin/pyvenv-3.5 $ /tmp/venv-1/bin/python /bin/pyvenv-3.5 /tmp/venv-2 --system-site-packages $ ls -l /tmp/venv-2/bin/python* … /tmp/venv-2/bin/python -> /tmp/venv-1/bin/python … /tmp/venv-2/bin/python3 -> python $ ls /tmp/venv-2/bin/ activate activate.fish easy_install-3.5* pip3* python@ activate.csh easy_install* pip* pip3.5* python3@ $ /tmp/venv-2/bin/python -m site sys.path = [ '/tmp', '/usr/lib64/python35.zip', '/usr/lib64/python3.5', '/usr/lib64/python3.5/plat-linux', '/usr/lib64/python3.5/lib-dynload', '/tmp/venv-2/lib64/python3.5/site-packages', '/tmp/venv-2/lib/python3.5/site-packages', '/usr/lib64/python3.5/site-packages', '/usr/lib/python3.5/site-packages', ] As you can see (last two entries), the system site-packages are taken from the real system installation, not from the stacked venv-1. This means, venvs can not be stacked. 3. Now let's see how guix-profile installed python works. I used a somewhat current HEAD (7e4e3df4e8) to ensure using the most current wrappers etc. ==================================================================== $ ./pre-inst-env guix package -i python … 3a. Do not set PYTHONPATH when setting up the venv. ---------------------------------------------------- $ ~/.guix-profile/bin/pyvenv-3.6 /tmp/venv-3a … $ /tmp/venv-3a/bin/python -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/venv-3a/lib/python3.6/site-packages', ] As expected there are only the venvs' site-packges in sys.path. 3b. Set PYTHONPATH when setting up the venv. ---------------------------------------------------- $ PYTHONPATH="$HOME/.guix-profile/lib/python3.6/site-packages" ~/.guix-profile/bin/pyvenv-3.6 /tmp/venv-3b … $ /tmp/venv-3b/bin/python -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/venv-3b/lib/python3.6/site-packages', ] Again there are only the venvs' site-packges in sys.path. This is excpected, since PYTHONPATH only effects the run of pyenv-3.6 4. Same as 3, but use --system-site-packages ==================================================================== 4a Don't set PYTHONPATH when setting up the venv. ---------------------------------------------------- $ ~/.guix-profile/bin/pyvenv-3.6 /tmp/venv-4a --system-site-package … $ /tmp/venv-4a/bin/python -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/venv-4a/lib/python3.6/site-packages', '/gnu/store/…-python-3.6.3/lib/python3.6/site-packages', ] This is *not* what what a Guix user would expect. For the Guix user's perspective his/her "Python site-packages" are those in $GUIX_PROFILE. Esp. since guix never installs into /gnu/store/…-python-3.6.3/lib/python3.6/site-packages and thus this path never contains additional "site" packages. 4b Set PYTHONPATH when setting up the venv. ---------------------------------------------------- $ PYTHONPATH="$HOME/.guix-profile/lib/python3.6/site-packages" ~/.guix-profile/bin/pyvenv-3.6 /tmp/venv-4b --system $ /tmp/venv-4b/bin/python -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/venv-4b/lib/python3.6/site-packages', '/gnu/store/…-python-3.6.3/lib/python3.6/site-packages', ] Result is the same as for 4a), reason as in 3b). 5. Would the venv-hack I posted a view days ago solve the issue? ==================================================================== 5a. Verify the venv-hack works ---------------------------------------------------- $ cp -r ~/.guix-profile /tmp/guix-profile $ mkdir !$ mkdir /tmp/guix-profile $ cp -r ~/.guix-profile/* /tmp/guix-profile $ echo 'include-system-site-packages = false' > /tmp/guix-profile/pyvenv.cfg $ /tmp/guix-profile/bin/python3 -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/guix-profile/lib/python3.6/site-packages', ] As expected, the profile's site-packages are included in sys.path. 5b Build a venv based on this (hacked) profile ---------------------------------------------------- $ /tmp/guix-profile/bin/pyvenv-3.6 /tmp/venv-5b --system-site-packages … $ /tmp/venv-5b/bin/python -m site sys.path = [ '/tmp', '/gnu/store/…-python-3.6.3/lib/python36.zip', '/gnu/store/…-python-3.6.3/lib/python3.6', '/gnu/store/…-python-3.6.3/lib/python3.6/lib-dynload', '/tmp/venv-5b/lib/python3.6/site-packages', '/gnu/store/…-python-3.6.3/lib/python3.6/site-packages', ] As in 4a, this is *not* what what a Guix user would expect. The profile's site-packages should be in sys.path, not /gnu/store/…-python-3.6.3/lib/python3.6/site-packages. -- Regards Hartmut Goebel | Hartmut Goebel | h.goe...@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |