On 2/13/21 5:51 PM, Zac Medico wrote: > On 2/10/21 11:11 AM, Rich Freeman wrote: >> On Wed, Feb 10, 2021 at 12:57 PM Andreas K. Hüttel <dilfri...@gentoo.org> >> wrote: >>> >>> * what portage features are still needed or need improvements (e.g. binpkg >>> signing and verification) >>> * how should hosting look like >> >> Some ideas for portage enhancements: >> >> 1. Ability to fetch binary packages from some kind of repo. > > The old PORTAGE_BINHOST functionality has been replaced with a > binrepos.conf file that's very similar to repos.conf: > > https://bugs.gentoo.org/668334 > > It doesn't have explicit support for multiple local binary package > repositories yet, but somebody got it working with src-uri set to a > file:/ uri as described in comments on this bug: > > https://bugs.gentoo.org/768957 > >> 2. Ability to have multiple binary packages co-exist in a repo (local >> or remote) with different build attributes (arch, USE, CFLAGS, >> DEPENDS, whatever). > > We can now enable FEATURES=binpkg-multi-instance by default now that > this bug is fixed: > > https://bugs.gentoo.org/571126 > >> 3. Ability to pick the most appropriate binary packages to use based >> on user preferences (with a mix of hard and soft preferences). > > Current package selection logic for binary packages is basically the > same as for ebuilds. These are the notable differences: > > 1) Binary packages are sorted in descending order by (version, mtime), > so then most recent builds are preferred when the versions are identical. > > 2) The --binpkg-respect-use option rejects binary packages what would > need to be rebuilt in order to match local USE settings. > >> One idea I've had around how #2-3 might be implemented is: >> 1. Binary packages already contain data on how they were built (USE >> flags, dependencies, etc). Place this in a file using a deterministic >> sorting/etc order so that two builds with the same settings will have >> the same results. > > This would only be needed to multi-profile binhosts that provide a > variety of configurations for the same package. > > Features like this are not necessary if the binhost only intends to > provide packages for a single profile. > >> 2. Generate a hash of the file contents - this can go in the filename >> so that the file can co-exist with other files, and be located >> assuming you have a full matching set of metadata. > > For FEATURES=binpkg-multi-instance we currently use an integer BUILD_ID > ensure that file names are unique. > >> 3. Start dropping attributes from the file based on a list of >> priorities and generate additional hashes. Create symlinked files to >> the original file using these hashes (overwriting or not existing >> symlinks based on policy). This allows the binary package to be found >> using either an exact set of attributes or a subset of higher-priority >> attributes. This is analogous to shared object symlinking. >> 4. The package manager will look for a binary package first using the >> user's full config, and then by dropping optional elements of the >> config (so maybe it does the search without CFLAGs, then without USE >> flags). Eventually it aborts based on user prefs (maybe the user only >> wants an exact match, or is willing to accept alternate CFLAGs but not >> USE flags, or maybe anything for the arch is selected> 5. As always the >> final selected binary package still gets evaluated >> like any other binary package to ensure it is usable. >> >> Such a system can identify whether a potentially usable file exists >> using only filename, cutting down on fetching. In the interests of >> avoiding useless fetches we would only carry step 3 reasonably far - >> packages would have to match based on architecture and any dynamic >> linking requirements. So we wouldn't generate hashes that didn't >> include at least those minimums, and the package manager wouldn't >> search for them. >> >> Obviously you could do more (if you have 5 combinations of use flags, >> look for the set that matches most closely). That couldn't be done >> using hashes alone in an efficient way. You could have a small >> manifest file alongside the binary package that could be fetched >> separately if the package manager wants to narrow things down and >> fetch a few of those to narrow it down further. > > All of the above is oriented toward multi-profile binhosts, so we'll > have to do a cost/benefit analysis to determine whether it's worth the > effort to introduce the complexity that multi-profile binhosts add.
This idea to borrow paludis's notion of pbins could work well for a multi-profile binhost: https://archives.gentoo.org/gentoo-dev/message/1aea0de0ff588c67bf652f4c1f8ef304 -- Thanks, Zac
signature.asc
Description: OpenPGP digital signature
