Hey Donald! As far as using pip to do stuff system-wide, I wrote thoughts on http://notes.pault.ag/debian-python
As for the rest of it, distutils is actually concretely shitty, and replacing it with setuptools sounds sane. Sounds like a solid idea. Perhaps we can add a Lintian warning for using distutils; it's really nasty cruft, and it's going to bite the archive sooner rather than later. Cheers, Paul On Thu, Jan 21, 2016 at 9:19 AM, Donald Stufft <don...@stufft.io> wrote: > Hello! > > I'd like to suggest a change to the Debian Policy around Python packages > that > will help enable the world of Python packaging to continue to progress > forward. > > First, a little bit of background: > > At the Python level there are three metadata formats for Python packaging: > > * The original, setuptools style .egg-info directories. > * The distutils style .egg-info *file* added to distutils at some point. > * The new and improved, wheel based .dist-info directories. > > The presence of any of these files will signal to Python tools that a > particular distribution has been installed, however there are two fairly > major > and important differences between the distutils style, and the other two. > > 1. The distutils style has no provisions to record what files on the system > belong to the installed distribution, making it appear to Python tooling > that there *are* no files other than the metadata file itself. > > 2. The distutils style has no provisions to include additional metadata > files > in the metadata, making it impossible to extend the python level > metadata > with additional files. > > I have a series of improvements that I'd like to make to the packaging > toolchain that will sort of build on one another, but which is not going to > function correctly with the distutils style metadata and I'm hoping that I > can > convince y'all to make it policy to default to generating one of the other > two > kinds (with varying methods, more on that later). > > Concretely the thing that this is blocking right now, is that with the > newly > released pip 8.0 I tried to make it so that pip will refuse to uninstall a > project that is installed with distutils style metadata. This is because > we do > not have any way to associate the actual .py (and others) files on disk > with > the installed metadata, so all we have ever done is just simply remove the > metadata file, making it appear as if the item is uninstalled but leaving > behind all of the actual files. However I'm going to be reverting this in a > pip 8.0.1 release because it caused a decent amount of breakage amongst > pip's > users, almost all of them people who are attempting to upgrade OS provided > packages using pip. > > Now, I know that upgrading OS provided packages using pip is less than > optimal > and I would greatly prefer that people did not do it (and I'm generally in > agreement) however if we don't enable people to do it, they'll just > continue to > use an old version of pip and file bugs. It's a non starter for pip to > make it > impossible to do. > > In addition to the uninstall bit, it also means that things like pip show > -f > return junk information for packages installed in this way. > > Beyond just (eventually) enabling pip to disable uninstallations of > distutils > based installs this will start to allow some other future changes that I > think > will be more interesting to Debian. The uninstallation of distutils based > installs comes hand in hand with pip stomping all over already existing > files > willy nilly because the way upgrading a project like that works is pip > uninstalls the metadata file that says X is installed, then it just > overwrites > over any of the files that happen to be in it's way when it installs the > newer > version. If we can remove the need for pip to gleefully overwrite files to > support these types of installed packages, then we can make it so pip will > hard fail if it attempts to overwrite an already existing file on disk. > > An additional benefit here is that by switching to using the directory > based > options, we can add additional metadata files to the installed projects, > much > like the INSTALLER file from PEP376 (IIRC). This file will likely be the > path > to having pip refuse to touch OS owned files all together without some > sort of > --force flag to override the safety switch. > > As far as compatibility goes, pip has always forced everything to be > installed > using setuptools and as far as I am aware, there's no real fallout from > doing > so. I think in 2016 it's pretty reasonable to assume that a Python project > is > capable of being installed using setuptools instead of distutils. > > So without getting into the actual *method* of doing this (of which there > are > several different options with different trade offs) does this sound like > something at all that Debian would be interested in? > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > -- :wq
