> On 14 Jul 2014, at 08:08, Gregory Haynes <g...@greghaynes.net> wrote:
> 
>  
>> On Sun, Jul 13, 2014, at 02:36 PM, James Polley wrote:
>>  
>> We're also thinking about how we continue to offer the pre-built wheels
>> for each of our build platforms. For infra, what I'm thinking is:
>>  
>> On each mirror slave (We have one for each OS combo we use), do
>> something similar to:
>>  
>> pip wheel -r global-requirements.txt
>> rsync $wheelhouse pypi.openstack.org/$(lsb_release)
>>  
>> This may require keeping pypi-mirror and using an option to only do
>> wheel building so that we can get the directory publication split. Ok. I
>> got bored and wrote that:
>>  
>> https://review.openstack.org/106638
>>  
>> So if we land that, you can do;
>>  
>> pip wheel -r global-requirements.txt
>> run-mirror --wheels-only --wheelhouse=wheelhouse --wheeldest=mirror
>> rsync -avz mirror pypi.openstack.org:/srv/mirror
>>  
>> If we went the devpi route, we could do;
>>  
>> pip wheel -r global-requirements.txt
>> for pkg in $wheelhouse; do
>>   devpi upload $pkg
>> done
>>  
>> And put that into a cron.
>>  
>> Obviously "keeping pypi-mirror" would require the least amount of change to 
>> how we suggest developers set up their systems.
>>  
>> I think the devpi option seems fairly reasonable too. It looks like it's 
>> easier (and faster, and less bandwidth-consuming) than setting up 
>> bandersnatch or apt-mirror, which we currently suggest people consider. It 
>> doesn't look any more heavyweight than having a squid proxy for caching, 
>> which we currently suggest as a bare minimum.
>>  
>> For an individual dev testing their own setup, I think we need a slightly 
>> different approach from the infra approach listed above though. I'm assuming 
>> that it's possible to probe the package index to determine if a wheel is 
>> available for a particular version of a package yet. If that's the case, we 
>> should be able to tweak tools like os-svc-install to notice when no wheel is 
>> available, and build and upload the wheel.
>>  
>> I think this should give us a good balance between making sure that each 
>> build (except the first) uses wheels to save time, still gets the latest 
>> packages (since the last time the system was online at least), and the user 
>> doesn't need to remember to manually update the wheels when they're online.
>  
> This gave me an idea:
> There was talk about pip being able to use a wheel cache (wheelhouse). Can we 
> bind-mount an arch-specific wheelhouse from the hypervisor into our chroots 
> as we build? This would let people get most of the wheel speedup while doing 
> almost no specifal configuration.

Pip still doesn't handle case insensitivity on file:// indexurls as well as its 
does for http:// - that should be fixed with a 1.6.0 release. There's some 
chance that we'll run into issues with this that we wouldn't hit using http, 
but i don't expect any major issues.

This still leaves us needing to build that arch-specific wheelhouse though


>  
> -Greg
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to