On 04/15/2015 09:50 PM, Robert Collins wrote: > On 16 April 2015 at 11:59, Sean Dague <s...@dague.net> wrote: > >> I think the completeness statement here is as follows: >> >> 1. For OpenStack to scale to the small end, we need to be able to >> overlap services, otherwise you are telling people they basically have >> to start with a full rack of hardware to get 1 worker. > > I don't see how that follows: you can run N venvs on one machine. Its > basically a lighter-weight container than containers, just without all > the process and resource isolation. > >> 2. The Linux Distributors (who are a big part of our community) install >> everything at a system level. Python has no facility for having multiple >> versions of libraries installed at the system level, largely because >> virtualenvs were created (which solve the non system application level >> problem really well). > > Actually, the statement about Python having no facility isn't true - > there are eggs and a mechanism to get a specific version of a > dependency into a process. It's not all that widely used, largely > because of the actual things thats missing: Python *doesn't* have the > ability to load multiple versions of a package into one process. So > once you ask for testtools==1.0.0, that process has only > testtools-1.0.0 in the singleton sys.modules['testtools'], and the > import machinery is defined as having the side effect of changing > global state, so this is sufficiently tricky to tackle noone has, even > with importlib etc being around now. NB: 'vendoring' does something in > this space by shifting an import to a different location, but its > fragile. > >> 3. The alternative of putting a venv inside of every service violates >> the principle of single location security update. > > So does copying code around, but we've officially adopted that as our > approach-until-things-are-mature... and in fact we're talking cluster > software, so there is (except the small scale) absolutely no > expectation of single-location security updates: we know and expect to > have to update N==machines locations, making that M>N is a small > matter of automation, be that docker/lxc/venvs. > > I think the arguments about devstack and ease of hacking, + our > deployer community specifically requesting it are sufficient. Whether > they need to request it or not, they have :). > >> Note: there is an aside about why we might *not* want to do this.... >> >> That being said, if you deploy at a system level your upgrade unit is >> now 1 full box, instead of 1 service, because you can't do library >> isolation between old and new. A worker might have neutron, cinder, and >> nova agents. Only the Nova agents support rolling upgrade (cinder is >> working hard on this, I don't think neutron has visited this yet). So >> real rolling upgrade is sacrificed on this alter of install everything >> at a system level. > > Yup. And most deployment frameworks want to scale by service, not by > box, which makes genuine containers super interesting.... > >>> Concretely, devstack should be doing one pip install run, and in >>> stable branches that needs to look something like: >>> >>> $ pip install -r known-good-list $path_to_nova $path_to_neutron .... >> >> Also remember we need to -e install a bunch of directories, not sure if >> that makes things easier or harder. > > There's a particular bit of ugly in pip where directories have to be > resolved to packages, but if we use egg fragment names we can tell pip > the name and avoid that. The thing that that lookup does is cause > somewhat later binding of some requirements calls - I'm not sure it > would be a problem, but if it is its fairly straight forward to > address. > >> So, one of the problems with that is often what we need is buried pretty >> deep like - >> https://github.com/openstack-dev/devstack/blob/master/lib/nova_plugins/functions-libvirt#L39 > > Both apt and rpm will perform much faster given a single invocation > than 10 or 20 little ones. There's a chunk of redundant work in dpkg > itself for instance that can be avoided by a single call. So we might > want to do that for that reason alone. pip doesn't currently do much > global big-O work, so it shouldn't be affected, but once we do start > considering already-installed-requirements, then it will start to have > the same issue. > >> If devstack needs to uplift everything into an install the world phase, >> I think we're basically on the verge of creating OpenStack Package >> Manager so that we can specify this all declaratively (after processing >> a bunch of conditional feature logic). Which, is a thing someone could >> do (not it), but if we really get there we're probably just better off >> thinking about reviving building debs so that we can have the package >> manager globally keep track of all the requirements across multiple >> invocations. > > mmm, I don't see that - to me a package manager has a lot more to do > with dependencies and distribution- I'd expect an opm to know about > things like 'nova requires the optional feature X to work with dvr, or > Y to work with ovn'. And where to get the tarballs or git repos from > given just 'openstack/nova'. Refactoring devstack into a bunch of pure > code - calculate the needed binaries and python bits, and a bunch of > impure code - do the installs, write the configs, start the services - > seems straight forward, good for maintenance, and a pretty low bar. > Maybe even migrate some of it to Python at the same time :).
Possibly, the devil is in the details. Also it means it won't play nice with manual pip installs before / after, which are sometimes needed. Mostly I find it annoying that pip has no global view, so all tools that call it have to construct their own global view and only ever call pip once to get a right answer. It feels extremely fragile. It's also not clear to me how easy it's going to be to debug. pip install .... <150 packages> <10 directories> ... Version conflict A>=1.4, A<1.3. (For reference in http://logs.openstack.org/42/173842/1/check/check-tempest-dsvm-full/8b0b195/logs/devstacklog.txt.gz we install 192 pip packages & 10 directories from pip install -e & 190 deb packages from apt. The pip packages will increase once we remove some additional python deb packages. -Sean -- Sean Dague http://dague.net __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
