I agree with Flavio. This looks really cool, and I had a very similar idea recently. I'll try to find some time to give this a whirl.
-Dave On Fri, Sep 5, 2014 at 10:37 AM, Flavio Percoco <fla...@redhat.com> wrote: > On 09/05/2014 04:21 PM, Monty Taylor wrote: > > On 09/05/2014 06:32 AM, Sean Dague wrote: > >> While reviewing this zookeeper service group fix in Nova - > >> https://review.openstack.org/#/c/102639/ it was exposed that the > >> zookeeper tests aren't running in infra. > >> > >> The crux of the issue is that zookeeper python modules are C extensions. > >> So you have to either install from packages (which we don't do in unit > >> tests) or install from pip, which means forcing zookeeper dev packages > >> locally. Realistically this is the same issue we end up with for mysql > >> and pg, but given their wider usage we just forced that pain on > >> developers. > >> > >> But it seems like a bad stand off between testing upstream and testing > >> normal path locally. > >> > >> Big picture it would be nice to not require a ton of dev libraries > >> locally for optional components, but still test them upstream. So that > >> in the base case I'm not running zookeeper locally, but if it fails > >> upstream because I broke something in zookeeper, it's easy enough to > >> spin up that dev env that has it. > >> > >> Which feels like we need some decoupling on our requirements vs. tox > >> targets to get there. CC to Monty and Clark as our super awesome tox > >> hackers to help figure out if there is a path forward here that makes > >> sense. > > > > Funny story - I've come to dislike what we're doing here, so I've been > > slowly working on an idea in this area: > > > > https://github.com/emonty/dox > > > > The tl;dr is "it's like tox, except it uses docker instead of > > virtualenv" - which means we can express all of our requirements, not > > just pip ones. > > > > It's not quite ready yet - although I'd be happy to move it in to > > stackforge or even openstack-dev and get other people hacking on it with > > me until it is. The main problem that needs solving, I think, is how to > > sanely express multiple target environments (like py26,py27) without > > making a stupidly baroque config file. OTOH, tox's insistence of making > > a new virtualenv for each "environment" is heavyweight and has led to > > some pretty crazy hacks across the project. Luckily, docker itself does > > an EXCELLENT job at handling caching and reuse - so I think we can have > > a set of containers that something in infra (waves hands) publishes to > > dockerhub, like: > > > > infra/py27 > > infra/py26 > > > > And then have things like nova build on those, like: > > > > infra/nova/py27 > > > > Which would have zookeeper as well > > > > The _really_ fun part, again, if we can figure out how to express it in > > config without reimplementing make accidentally, is that we could start > > to have things like: > > > > infra/mysql > > infra/postgres > > infra/mongodb > > > > And have dox files say things like: > > > > Nova unittests want a python27 environment, this means we want an > > infra/mysql container, an infra/postgres container and for those to be > > linked to the infra/nova/py27 container where the tests will run. > > > > Since those are all reusable, the speed should be _Excellent_ and we > > should be able to more easily get more things runnable locally without > > full devstack. > > > > Thoughts? Anybody wanna hack on it with me? I think it could wind up > > being a pretty useful tool for folks outside of OpenStack too if we get > > it right. > > > > I think it is sexy - I don't describe ideas/software as sexy that often > but this one deserves it. I'm interested in helping out. > > I'll clone it and give it a try - or at least take a look at it. > > Flavio > > > Monty > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > -- > @flaper87 > Flavio Percoco > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- David Shrewsbury (Shrews)
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
