> On 14 Jul 2014, at 08:08, Gregory Haynes <g...@greghaynes.net> wrote: > > >> On Sun, Jul 13, 2014, at 02:36 PM, James Polley wrote: >> >> We're also thinking about how we continue to offer the pre-built wheels >> for each of our build platforms. For infra, what I'm thinking is: >> >> On each mirror slave (We have one for each OS combo we use), do >> something similar to: >> >> pip wheel -r global-requirements.txt >> rsync $wheelhouse pypi.openstack.org/$(lsb_release) >> >> This may require keeping pypi-mirror and using an option to only do >> wheel building so that we can get the directory publication split. Ok. I >> got bored and wrote that: >> >> https://review.openstack.org/106638 >> >> So if we land that, you can do; >> >> pip wheel -r global-requirements.txt >> run-mirror --wheels-only --wheelhouse=wheelhouse --wheeldest=mirror >> rsync -avz mirror pypi.openstack.org:/srv/mirror >> >> If we went the devpi route, we could do; >> >> pip wheel -r global-requirements.txt >> for pkg in $wheelhouse; do >> devpi upload $pkg >> done >> >> And put that into a cron. >> >> Obviously "keeping pypi-mirror" would require the least amount of change to >> how we suggest developers set up their systems. >> >> I think the devpi option seems fairly reasonable too. It looks like it's >> easier (and faster, and less bandwidth-consuming) than setting up >> bandersnatch or apt-mirror, which we currently suggest people consider. It >> doesn't look any more heavyweight than having a squid proxy for caching, >> which we currently suggest as a bare minimum. >> >> For an individual dev testing their own setup, I think we need a slightly >> different approach from the infra approach listed above though. I'm assuming >> that it's possible to probe the package index to determine if a wheel is >> available for a particular version of a package yet. If that's the case, we >> should be able to tweak tools like os-svc-install to notice when no wheel is >> available, and build and upload the wheel. >> >> I think this should give us a good balance between making sure that each >> build (except the first) uses wheels to save time, still gets the latest >> packages (since the last time the system was online at least), and the user >> doesn't need to remember to manually update the wheels when they're online. > > This gave me an idea: > There was talk about pip being able to use a wheel cache (wheelhouse). Can we > bind-mount an arch-specific wheelhouse from the hypervisor into our chroots > as we build? This would let people get most of the wheel speedup while doing > almost no specifal configuration. Pip still doesn't handle case insensitivity on file:// indexurls as well as its does for http:// - that should be fixed with a 1.6.0 release. There's some chance that we'll run into issues with this that we wouldn't hit using http, but i don't expect any major issues. This still leaves us needing to build that arch-specific wheelhouse though > > -Greg > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev