On 21 January 2015 at 21:23, Alec Ten Harmsel <a...@alectenharmsel.com> wrote:
>
> On 01/21/2015 07:47 AM, Sam Bishop wrote:
>> So I've been thinking crazy thoughts.
>>
>> Theoretically it can't be that hard to do a complete package binhost for
>> gentoo.
>
> I love that you qualify this with "theoretically."
>
I'm in a position where the cost of these servers may become less than
the cost of paying developers to wait while ebuilds compile. So II'm
having a semi-serious theoretical discussion with myself as to the
merits of opening this up to the entire Gentoo community and a much
more serious theoretical discussion here right now with anyone on this
list as to just how would one do this.
>>
>> To be clear, when i say complete, Im referring to building, all
>> versions of all ebuilds marked stable or unstable on amd64, with every
>> combination of use flags.
>
> Every ebuild with every combination of USE flags? This is likely
> impossible, and definitely not feasible. With 17000ish ebuilds in the
> portage tree and assuming each only has 2 USE flags, this would be
> building 17000*2^2 = 68,000 packages. If average build time is 20
> seconds (nice server w/ SSD and enough RAM to build in /tmp), it'd take
> 377ish hours to do an initial build of the tree. I guess this isn't so
> bad. Of course, there are outliers like www-client/firefox: 19
> non-language USE flags, so 2^19 different firefox permutations at a fast
> 5 minutes apiece would take 43000 hours. I haven't looked at
> REQUIRED_USE, so there could be less than 2^19 different combinations of
> flags; taking it down to 2^10 combinations is only 85 hours or so.
>
Or... looking at it another way, in order to do this in under 24 hrs,
the initial burst capacity would need to be, using your time estimate
and a healthy over estimate of capacity. It would need approximately
20 'nice servers'... for a day for the initial build, then a much
reduced number in order to continue the ongoing work of building all
the new changes.
>>
>> This pretty much boils down to bytes and bytes of storage + compute
>> resources. Both of which are easily available to me. So I began
>> pondering and here I am, thinking to myself "is this really all there
>> is too it"?
>
> A full CentOS mirror is ~600GB iirc, so you're gonna need a ton of storage.
>
1TB on AWS S3 costs me $30 ... thats about 20 minutes of developer
time saved to pay back the cost.
At the moment our build pipeline can take over 45 minutes... most of
it is ebuilds compiling so it won't be hard to speed up with a
binhost.
Were not exactly going to build 'less often', so this does add up.
>> Does it really come down to CPU cycles and repeatedly running through
>> the following commands for each combination of ebuild, version and use
>> flags
>> emerge --emptytree --onlydeps ${name}
>> emerge --emptytree --buildpkgonly --buildpkg ${name}
>>
>> Obviously running them in a clean environment each time, either by
>> chroot or other means.
>> Then just storing the giant binhost somewhere suitable such as an AWS
>> s3 bucket setup to work via HTTP so the normal tools work fine with
>> it.
>>
>
> I haven't used binpkgs in a long time, but I think USE on the client
> machine has to match the USE of the package being installed. Managing
> all of this would be a nightmare unless you wrote your own special
> portage server that served up binpkgs in a USE-aware way and a portage
> host could request a binpkg with a certain USE.
>
> Theoretically, great idea. I think this would be possible if you had
> maybe 3 or 4 different USE combos (i.e. one for servers, one for KDE
> client machines, one for gnome clients, etc.).
>
> Alec
>
> P.S. I'm reasonably sure my math is correct, but I would appreciate
> corrections.
>
I don't see why it can't be all the combinations, the issue is
storage, and the storage costs could be a lot lower than expected
given how hard it is to guess. So I would also love to see some
corrected/more accurate estimates, especially any that are based on
numbers from anyone who has been involved in running a tinderbox,
these aren't exactly numbers many people have sitting around haha.
