On 01/17/2015 04:46 PM, Patrick Lauer wrote: > On Saturday 17 January 2015 13:12:56 Zac Medico wrote: >> On 01/17/2015 03:35 AM, Patrick Lauer wrote: >>> * Portage is too slow >>> >>> On 'small' hardware emerge -upNDv @world can take enough time >>> to make updates prohibitive - on an 800Mhz machine it took me >>> about 3 days to figure out a solution to some silly blockers due to >>> the >>> very slow change test cycle >>> Fix: Do some profiling and un-refactoring to remove useless code >>> Fix: Set up a reference system (CI) to catch future regressions >> >> I'm certainly in favor of improving portage performance. However, for >> "small" hardware (which includes many ARM and MIPS systems), we really >> need to focus on binary package support. Note that dependency resolution >> for installing binary packages tends to be much simpler than for >> building ebuilds from source, and this translates into much shorter >> dependency resolution time. >> > That's an orthogonal problem. And that's coming from someone who extensively > uses binpkgs already ...
Sure, but building from source on "small" hardware is not necessarily the best practice. If our binary package support was better, then people might be more likely to use "big" hardware to build packages for distribution to "small" hardware. > The problem with (dependency resolution) performance is that in some > scenarios, even on fast machines, it takes "too long" - especially if there's > some silly useflag mismatch and two packages that have ~arch keywords, and > now > you need 12 attempts to figure out a solution that works for you ... > At 30 seconds for each resolution cycle that's 6 minutes waiting for portage. > > If it were faster it'd almost be interactive ... Yeah, that would be great. On a related note, I think that the default emerge --backtrack=10 setting is too high, causing emerge to waste lots of cpu time before it ultimately fails to find a valid solution. I've filed a bug to make it default to --backtrack=3 [1]. > Also - just for comparison: > On my currently fastest box "emerge -ep @world" takes about 3 seconds since > there's almost no packages installed. > On my currently slowest box same takes about 15 minutes, because (1) lots of > packages installed and (2) slow CPU and (3) brutally slow IO > > Binpkgs only help so much - if any dependencies change I need to sync the > configuration between client and server, and to see if it resolves I need to > do > a dryrun on the client (or be very certain that the configuration really > matches) - or risk that it's going to fail because of mismatches. For some, maybe a nice way to sync configuration would be to create a customized profile. Then, emerge --sync would pull in your configuration updates from the server. With "profile-formats = portage-2" in metadata/layout.conf of your profiles repository, you can put things like "gentoo:default/linux/amd64/13.0/desktop" in the parent file of your profile, so it inherits from a profile in the "gentoo" repository. > I haven't quite figured out why portage needs such humongous amounts of > memory > (>300MB ?!) Yes, I would like to work on reducing the memory consumption. > and why it needs to spend a minute to figure out how to install a > simple almost-no-dependencies tool like htop or iotop. Note that the emerge --complete-graph-if-new-ver and --complete-graph-if-new-use options are enabled by default. These options will force emerge to create a complete dependency graph, in order to ensure that all reverse-dependencies are respected. > There's some obvious > badness, and just saying "well let's use binpkgs" won't fix it (and, well, on > the binpkg server if you have 3k packages installed you get the same > performance issues, so ignoring the issue is kinda not a good idea) Of course not. > There's been some good progress, I remember you reducing memory usage already > (>500MB -> 350MB or so for merging kernel sources) and there's some speedups > from the last round of performance work. Still I think we can do a lot better > :) Sure we can. > Thanks for your efforts, > > Patrick [1] https://bugs.gentoo.org/show_bug.cgi?id=536926 -- Thanks, Zac
