This is a bugbear that keeps cropping up and biting us. I'm hoping we can figure out a permanent fix.
The problem that occurs is the result of a few interacting things: - requests has very very specific versions of urllib3 it works with. So specific they aren't always released yet. - Linux vendors often unbundle urllib3 from requests and then apply what patches were needed to their urllib3; while not updating their requests package dependencies to reflect this. - we use urllib3 in some places and requests in others (but we don't mix them up) - if for any reason we have a distro-altered requests + a pip-installed urllib3, requests will [usually] break... see the 'not always released yet' key thing above. Now, there are lots of places this last thing can happen; they all depend on us having a dependency on requests that is compatible with the version installed by the distro, but a urllib3 dependency that triggers an upgrade of just urllib3. When constraints are in use, the requests version has to match the distro requests version exactly, but that will happen from time to time. e.g. - dvsm test jobs where the base image already has python-requests installed in it - virtualenvs where system-site-packages are enabled There are a few strategies that have been proposed to fix this. AIUI they are: - make sure none of our testing environments include distro requests packages. - make our requirements be tightly matched to what requests needs to deal with the unbundling - teach pip how to identify and avoid this situation by always upgrading requests (even if thats just a re-install of the version from PyPI) when the installed requests is a distro installed version **and** urllib3 is being modified. - get the distros to stop un-vendoring urllib3 The first one addresses the situation for the CI gate but doesn't avoid developers getting bitten on their local machines. And installing any distro thing that uses requests would re-instate the potential for breakage. So while its not harmful, I don't think its sufficient to make this go away. The second is trivially insufficient - anytime requests vendored urllib3 is not precisely identical to a released urllib3, it becomes impossible to satisfy that via dependency version pinning - the only way to satisfy it is with the urllib3 in the distro that has whatever change was needed included. The third approach will require some negotiation I suspect - because its aesthetically wrong: from an upstream perspective urllib3 is independent of requests, and vice-versa, but from a distro perspective they are tightly coupled, no variation permitted. The fourth approach meets the stone wall of 'but security' and 'no redundancy permitted' - I don't have the energy to try and get through the near-religious mindset I've encountered there before, though hey - if Fedora and Debian and Ubuntu folk are all interested in figuring out a sustainable way forward, that would be great: please don't feel cut out, I'm just not expecting anything. If there are other approaches, great - please throw them up here. -Rob -- Robert Collins <rbtcoll...@hp.com> Distinguished Technologist HP Converged Cloud __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
