A follow up. I've done a fair bit of prototyping a solution, and with Donald's gracious help and feedback, I think I have a plan that will work and should be compliant with policy. I'm beginning to make changes to various DPMT packages to build the whole stack, but I'm only uploading the simpler changes for now until I have a working local testbed.
The basic approach does involve building a bunch of wheels as explained earlier. We have to build the pip and setuptools wheels from our devendorized packages, and install those into the venv's site-packages directory. Installation is done by ensurepip, and I have a branch of python3.4 that re-enables this. Now that python3-wheel has cleared NEW, I need to modify the d/rules files to build the wheels. I have this working for pip in svn (along with significant packaging changes to use pybuild and update to 1.5.5), but I won't upload this just yet. It installs the wheels into /usr/share/python-wheels. The tricky bit is what to do about the recursively vendorized packages. Here I think the best course of action is as outlined previously - build wheels of these from quilt-patched Debian source at package build time, and install those too into /usr/share/python-wheels. When pyvenv is called, the appropriate wheels will be copied into <venv>/usr/share/python-wheels and the venv's pip will know to add those to sys.path. We cannot unpack these dependent wheels into the venv's site-packages so this is to me the simplest and best approach. I think Donald agrees. The one ugly bit is what to do about the venv wheels when you `pip upgrade pip` since this will pull the PyPI pip into the venv. Our unvendorized wheels won't get deleted from the venv, but they also won't be used (because PyPI's pip will bundle, and won't contain the patch to use the unvendored versions). IOW, our wheels will stick around as unused trash. Oh well - this only happens if you `pip upgrade pip` in the venv. (We can try to fix that later if we really care.) Other than extensive testing <wink>, what needs to happen now is that I need to modify the packaging of the dependencies to build the wheels. This is fairly easy for html5lib, requests, python3-chardet, and urllib3, although some of these need their setup.py patched to use setuptools instead of distutils.core (because only the former supports the bdist_wheel command provided by python3-wheel). These packages are all team maintained. It's trickier for distlib, colorama, and six because those are not team maintained, so I will generate patches and pester the maintainers once I've proven that the whole stack works. I've reached out to the colorama maintainer to see if we can move that into DPMT, since it's only ever seen NMUs since the initial upload. Colin, Matthias, keep an eye out for patches to six and distlib respectively. I won't finish this up this weekend, but will continue working on it early next week. Stay tuned, and best to ping me on IRC if you have questions. Cheers, -Barry
signature.asc
Description: PGP signature
