On Thu, Apr 05, 2018 at 01:27:13PM -0700, Clark Boylan wrote: > On Mon, Apr 2, 2018, at 9:13 AM, Clark Boylan wrote: > > On Mon, Apr 2, 2018, at 8:06 AM, Matthew Thode wrote: > > > On 18-03-31 15:00:27, Jeremy Stanley wrote: > > > > According to a notice[1] posted to the pypa-announce and > > > > distutils-sig mailing lists, pip 10.0.0.b1 is on PyPI now and 10.0.0 > > > > is expected to be released in two weeks (over the April 14/15 > > > > weekend). We know it's at least going to start breaking[2] DevStack > > > > and we need to come up with a plan for addressing that, but we don't > > > > know how much more widespread the problem might end up being so > > > > encourage everyone to try it out now where they can. > > > > > > > > > > I'd like to suggest locking down pip/setuptools/wheel like openstack > > > ansible is doing in > > > https://github.com/openstack/openstack-ansible/blob/master/global-requirement-pins.txt > > > > > > We could maintain it as a separate constraints file (or infra could > > > maintian it, doesn't mater). The file would only be used for the > > > initial get-pip install. > > > > In the past we've done our best to avoid pinning these tools because 1) > > we've told people they should use latest for openstack to work and 2) it > > is really difficult to actually control what versions of these tools end > > up on your systems if not latest. > > > > I would strongly push towards addressing the distutils package deletion > > problem that we've run into with pip10 instead. One of the approaches > > thrown out that pabelanger is working on is to use a common virtualenv > > for devstack and avoid the system package conflict entirely. > > I was mistaken and pabelanger was working to get devstack's USE_VENV option > working which installs each service (if the service supports it) into its own > virtualenv. There are two big drawbacks to this. This first is that we would > lose coinstallation of all the openstack services which is one way we ensure > they all work together at the end of the day. The second is that not all > services in "base" devstack support USE_VENV and I doubt many plugins do > either (neutron apparently doesn't?). > Yah, I agree your approach is the better, i just wanted to toggle what was supported by default. However, it is pretty broken today. I can't imagine anybody actually using it, if so they must be carrying downstream patches.
If we think USE_VENV is valid use case, for per project VENV, I suggest we continue to fix it and update neutron to support it. Otherwise, we maybe should rip and replace it. Paul > I've since worked out a change that passes tempest using a global virtualenv > installed devstack at https://review.openstack.org/#/c/558930/. This needs to > be cleaned up so that we only check for and install the virtualenv(s) once > and we need to handle mixed python2 and python3 environments better (so that > you can run a python2 swift and python3 everything else). > > The other major issue we've run into is that nova file injection (which is > tested by tempest) seems to require either libguestfs or nbd. libguestfs > bindings for python aren't available on pypi and instead we get them from > system packaging. This means if we want libguestfs support we have to enable > system site packages when using virtualenvs. The alternative is to use nbd > which apparently isn't preferred by nova and doesn't work under current > devstack anyways. > > Why is this a problem? Well the new pip10 behavior that breaks devstack is > pip10's refusable to remove distutils installed packages. Distro packages by > and large are distutils packaged which means if you mix system packages and > pip installed packages there is a good chance something will break (and it > does break for current devstack). I'm not sure that using a virtualenv with > system site packages enabled will sufficiently protect us from this case (but > we should test it further). Also it feels wrong to enable system packages in > a virtualenv if the entire point is avoiding system python packages. > > I'm not sure what the best option is here but if we can show that system site > packages with virtualenvs is viable with pip10 and people want to move > forward with devstack using a global virtualenv we can work to clean up this > change and make it mergeable. > > Clark > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
