On 6/20/24 11:29 AM, Peter Humphrey wrote:
On Thursday, 20 June 2024 14:40:12 BST Michael wrote:
On Thursday, 20 June 2024 14:27:18 BST Jack wrote:
On 6/20/24 8:46 AM, Peter Humphrey wrote:
While building a new KDE system (see my post a few minutes ago), I'm
finding the system stalling because it can't handle all its install
jobs. I have this set:
$ grep '\-j' /etc/portage/make.conf
EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]"
I don't know how much it would matter, but are you missing a number
after --jobs?
Without a number of jobs specified in make.conf emerge will not limit the
number of packages it tries to build, except it will not start new jobs
while there are at least --load-average=30 running already.
MAKEOPTS="-j16 -l16"
We went through all this at great length not long ago (months, perhaps: a
certain A. McK had returned to the list for a while). /usr/bin/make will stop
spawning make jobs once either (a) the number it's running reaches -j16 or (b)
the load average of those reaches -l16. Portage sending more tasks to
/usr/bin/make simply fills the latter's input queue.
Again, I don't know if it matters in this case, but my understanding is
that MAKEOPTS only affects jobs using make. I don't know if there are
equivalent controls for ninja or other build systems. Might that be
relevant here? If you run top, limit to running jobs and show the full
command, does that give any hints?
The CPU has 24 threads and 64GB RAM, and lots of swap space, and those
values have worked well for some time. Now, though, I'm going to have to
limit the --jobs or the --load-average.
On interrupting one such hang, I found that 32 install jobs had been
waiting to run; is this limit hard coded?
It's certainly a suspicious number.
I take it the --load-average is what it says, an average, so it will jump
above the specified number if you have not limited the --jobs number.
See above re. input queue.
I also saw "too many jobs" or something, and "could not read job
counter".
Is it now bug-report time?
You could set up a swap file, to avoid OOM situations, while you're tweaking
the --jobs & --load-average.
The existing 64GiB swap partition is rarely touched, if ever. I've never seen
an OOM error. I haven't touched jobs or loads for many months until today, nor
have I seen a failure to read a job counter.
Anyway, it still rankles that I can't use more than half the machine's power
because of limits in portage. This can't be the only 64GiB machine in gentoo-
land, surely.
