On 2018-03-24 00:03:12 +0000 (+0000), Simon McVittie wrote: [...] > On Fri, 23 Mar 2018 at 20:30:17 +0000, Jeremy Stanley wrote: > > This may also serve to help narrow down (via reverse dependency) the > > list of packages which will trigger violent reactions when mixed in > > the same context with pip 10 invocations, per > > https://github.com/pypa/pip/issues/4805 . > > Am I right in saying that there is a distutils limitation that means > the Python equivalent of "make uninstall" isn't reliable for packages > installed with distutils, and as a result, upstream developers want to > deprecate distutils?
More that pip can't reliably uninstall packages previously installed via distutils due to insufficient provided metadata. While I personally think that it's a bad idea pip would even attempt to uninstall (through removal of the underlying files) packages it wasn't previously managing when it's been asked (perhaps through a transitive dependency relationship) to upgrade them, it's unfortunately been the case for a while now that it does that. As far as I understand this is owing to RH-based distros using the same tree for RPM-installed and pip-installed packages rather than splitting the latter out to /usr/local like Debian derivatives, so it's significantly more likely that random files get left behind if pip can't uninstall them properly on an upgrade. Based on previous threads on the distutils-sig ML it doesn't sound like it's destined for general deprecation any time soon as it's part of the build toolchain for cpython itself. However the Python packaging ecosystem has been strongly discouraging its use outside of packaging the interpreter for some time and the same people are interested in seeing `setup.py install` type workflows disappear altogether, even in distro-packaged modules. > While deprecating libraries that have unfixable problems is a reasonable > goal, I'm reasonably sure that installing a library with apt/dpkg and > uninstalling it with "sudo pip uninstall" is not something we (should) > aim to support? Or am I misunderstanding? It's more that (and again, I'm not advocating for this pattern in general, though sometimes it can make sense in "dirty" environments like a throwaway chroot or ephemeral test system) when your distro has installed Python module "A" (perhaps because it's a dependency of an essential tool on that distro) and then you attempt to `sudo pip install ...` some module "B" into the same context but it has a direct or transitive dependency on "A" perhaps wanting a slightly newer version, pip will want to uninstall the existing "A" as part of upgrading it. The hope expressed by pip maintainers is that by changing the current warning about its inability to uninstall "A" (if the distro package was distutils-based) to a hard failure, they can get users relying on that to apply pressure on their respective distributions to change to newer packaging standards. > (Packages that were merely built with distutils won't normally depend > on distutils in any case, only build-depend.) Yes, apologies, that's what I meant. It's still _something_ we can query on to know whether a given binary package will present a problem in the face of a `sudo pip install ...` scenario which might try to upgrade it. > I thought the upstream Python maintainers strongly objected to the idea > of us installing something that claims to be Python but doesn't have > the complete Python standard library (in the interests of having Python > mean something predictable between OSs), but perhaps that information is > out of date and doesn't apply to distutils any more? My take is that "upstream Python" (who are not necessarily upstream of the popular Python package ecosystem tools, keep in mind) would probably rather Debian didn't exclude distutils from the stdlib for compatibility reasons, but the preference of the maintainers of Python packaging ecosystem tools is that distros cease using distutils to package up independent Python modules. -- Jeremy Stanley
