New submission from Eryk Sun <eryk...@gmail.com>:
In 3.7.2 on Windows, venv now uses a redirecting launcher 'script' for
python[w].exe. This replaces (on Windows only) the setup requirement to copy or
symlink the interpreter binaries (i.e. python[w].exe, python37.dll, and
vcruntime140.dll). Apparently this is required to be able to install Python as
a Windows Store app. I haven't experimented with it yet, so I'll just accept
that as a given. I'm curious to find out whether using symlinks would still
work, and even if it doesn't work for the app installation, whether we could
still support that option for desktop installations. I've used `--symlinks` for
a while now, since I grant the required privilege to the authenticated users
group and thus don't have to elevate to create symlinks. (I know it's not so
easy for many Windows users.)
The new launcher reads pyvenv.cfg to locate and execute the real python.exe.
Since the process image is no longer in the virtual environment's "Scripts"
directory (which has various consequences), we need a way to communicate the
launcher's path to make Python use the virtual environment. A -X command-line
option could work, but then all packages and tools that spawn worker processes,
such as multiprocessing, would need to be updated to look for this option in
sys._xoptions and propagate it. Instead the launcher sets a special
"__PYVENV_LAUNCHER__" environment variable. This is reasonable because
processes are usually created with a copy of the parent's environment. Some
environment variables may be added or modified, but it's rare for a child
process to get a completely new environment. (One example of the latter would
be creating a process that runs as a different user.)
An oversight in the current ecosystem is that py.exe and the distlib
entry-point launchers do not unset "__PYVENV_LAUNCHER__". Thus, when executing
a script from a virtual environment (e.g. either directly via py.exe or via the
.py file association), it will mistakenly be pinned into the virtual
environment if it runs in Python 3.7. Similarly, pip.exe for an installed
Python 3.7 will mistakenly install into the virtual environment. However, the
latter is out of scope here since the entry-point launchers are in distlib.
It's also a problem if we run the fully-qualified path for an installed Python
3.7, e.g. from shutil.which('python'). We can't automatically address this
since it's exactly the reason "__PYVENV_LAUNCHER__" exists. We have to know to
manually unset "__PYVENV_LAUNCHER__" in the environment that's passed to the
child. This should be documented somewhere -- maybe in the venv docs.
It's not a problem if a script runs unqualified "python.exe" since the final
result is usually the same, but for different reasons. With the launcher, it's
locked down by inheriting "__PYVENV_LAUNCHER__". With the previous design, the
application directory was the virtual environment's "Scripts" directory.
Unqualified "python.exe" was thus pinned for CreateProcessW, which checks the
application directory first. It's also not a problem if we run sys.executable,
since previously that was the virtual environment's executable. (In 3.7.2,
sys.executable gets set to the virtual environment's launcher, but that breaks
multiprocessing. See issue 35797.)
----------
components: Windows
messages: 334275
nosy: eryksun, paul.moore, steve.dower, tim.golden, vinay.sajip, zach.ware
priority: normal
severity: normal
stage: test needed
status: open
title: py.exe should unset the __PYVENV_LAUNCHER__ environment variable
type: behavior
versions: Python 3.7, Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue35811>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
