Re: [gentoo-user] Plasma 6
Thanks, I decided to uninstall, then do a sync and then install again but now getting lots of soft blocks. Think I'll wait until it's not in testing. John On Wed, 19 Jun 2024 17:47:49 -0500 Dale wrote: > jdm wrote: > > Hello, > > > > I've installed KDE Plasma 6 (plasma-meta) and all packages have > > built with no problem but whenever I start in wayland session it > > crashes out after a couple of seconds of logging in. X sessions > > works with no problems. > > > > So wondering if this is problem with just my PC or a more general > > issue and if wayland is ready yet for prime time. > > > > Thanks > > > > I recently updated on my newer rig. It started acting weird as well. > I thought it was me making changes in the kernel until I went back > to a kernel that I knew worked. What mine does is this. The monitor > seems to cut off and goes into the energy saving mode thing. I tried > restarting display-manager over ssh from my main rig and even that did > nothing. Then I tried something I didn't expect to work. I cut the > monitor off and back on. It came on but was at a console. I had > stopped display-manager so not to surprised really. I was trying all > sorts of options including trying to switch to a console. That could > have been how I ended up at a console. > > When it does that, power off your monitor and then power it back on. > See what it does. Even it is truly crashing, I doubt it will hurt > anything. Heck, it might actually come back on. Might give some guru > on this list a clue as to the cause or some reported bug. > > At the moment, I'm trying to downgrade. > > Dale > > :-) :-) > -- John D Maunder j...@jdm.myzen.co.uk
Re: [gentoo-user] Plasma 6
On Thursday, 20 June 2024 08:01:54 BST jdm wrote: > I decided to uninstall, then do a sync and then install again but now > getting lots of soft blocks. Think I'll wait until it's not in testing. I had just started building a new KDE system on my Ryzen M9 box, starting with no USE flags set, and only setting a flag when portage demanded it. I hadn't noticed that Qt6 was now out until I started installing plasma apps, when it became clear that some wholesale flag-setting was required. A few USE flags survived from the previous system, but I only needed these set for Qt6: kde-plasma/kwin lock dev-qt/*wayland kde-frameworks/*wayland kde-plasma/libplasmawayland media-libs/mesa wayland x11-base/xwayland libei There are other problems with multitasking, but I'll report those separately. -- Regards, Peter.
[gentoo-user] Too many simultaneous install jobs
Hello list, While building a new KDE system (see my post a few minutes ago), I'm finding the system stalling because it can't handle all its install jobs. I have this set: $ grep '\-j' /etc/portage/make.conf EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" MAKEOPTS="-j16 -l16" The CPU has 24 threads and 64GB RAM, and lots of swap space, and those values have worked well for some time. Now, though, I'm going to have to limit the --jobs or the --load-average. On interrupting one such hang, I found that 32 install jobs had been waiting to run; is this limit hard coded? I also saw "too many jobs" or something, and "could not read job counter". Is it now bug-report time? -- Regards, Peter.
Re: [gentoo-user] Too many simultaneous install jobs
On 6/20/24 8:46 AM, Peter Humphrey wrote: Hello list, While building a new KDE system (see my post a few minutes ago), I'm finding the system stalling because it can't handle all its install jobs. I have this set: $ grep '\-j' /etc/portage/make.conf EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" I don't know how much it would matter, but are you missing a number after --jobs? MAKEOPTS="-j16 -l16" The CPU has 24 threads and 64GB RAM, and lots of swap space, and those values have worked well for some time. Now, though, I'm going to have to limit the --jobs or the --load-average. On interrupting one such hang, I found that 32 install jobs had been waiting to run; is this limit hard coded? I also saw "too many jobs" or something, and "could not read job counter". Is it now bug-report time?
Re: [gentoo-user] Too many simultaneous install jobs
On Thursday, 20 June 2024 14:27:18 BST Jack wrote: > On 6/20/24 8:46 AM, Peter Humphrey wrote: > > Hello list, > > > > While building a new KDE system (see my post a few minutes ago), I'm > > finding the system stalling because it can't handle all its install jobs. > > I have this set: > > > > $ grep '\-j' /etc/portage/make.conf > > EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" > > I don't know how much it would matter, but are you missing a number > after --jobs? > > > MAKEOPTS="-j16 -l16" > > > > The CPU has 24 threads and 64GB RAM, and lots of swap space, and those > > values have worked well for some time. Now, though, I'm going to have to > > limit the --jobs or the --load-average. > > > > On interrupting one such hang, I found that 32 install jobs had been > > waiting to run; is this limit hard coded? I also saw "too many jobs" or > > something, and "could not read job counter". > > > > Is it now bug-report time? No, that's intended; it's what I meant about limiting it In fact I have now set --jobs=24; let's see how that goes. -- Regards, Peter.
Re: [gentoo-user] Too many simultaneous install jobs
On Thursday, 20 June 2024 14:27:18 BST Jack wrote: > On 6/20/24 8:46 AM, Peter Humphrey wrote: > > Hello list, > > > > While building a new KDE system (see my post a few minutes ago), I'm > > finding the system stalling because it can't handle all its install jobs. > > I have this set: > > > > $ grep '\-j' /etc/portage/make.conf > > EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" > > I don't know how much it would matter, but are you missing a number > after --jobs? Without a number of jobs specified in make.conf emerge will not limit the number of packages it tries to build, except it will not start new jobs while there are at least --load-average=30 running already. > > MAKEOPTS="-j16 -l16" > > > > The CPU has 24 threads and 64GB RAM, and lots of swap space, and those > > values have worked well for some time. Now, though, I'm going to have to > > limit the --jobs or the --load-average. > > > > On interrupting one such hang, I found that 32 install jobs had been > > waiting to run; is this limit hard coded? I take it the --load-average is what it says, an average, so it will jump above the specified number if you have not limited the --jobs number. > > I also saw "too many jobs" or > > something, and "could not read job counter". > > > > Is it now bug-report time? You could set up a swapfile, to avoid OOM situations, while you're tweaking the --jobs & --load-average. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Too many simultaneous install jobs
On Thursday, 20 June 2024 14:40:12 BST Michael wrote: > On Thursday, 20 June 2024 14:27:18 BST Jack wrote: > > On 6/20/24 8:46 AM, Peter Humphrey wrote: > > While building a new KDE system (see my post a few minutes ago), I'm > > > finding the system stalling because it can't handle all its install > > > jobs. I have this set: > > > > > > $ grep '\-j' /etc/portage/make.conf > > > EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" > > > > I don't know how much it would matter, but are you missing a number > > after --jobs? > > Without a number of jobs specified in make.conf emerge will not limit the > number of packages it tries to build, except it will not start new jobs > while there are at least --load-average=30 running already. > > > > MAKEOPTS="-j16 -l16" We went through all this at great length not long ago (months, perhaps: a certain A. McK had returned to the list for a while). /usr/bin/make will stop spawning make jobs once either (a) the number it's running reaches -j16 or (b) the load average of those reaches -l16. Portage sending more tasks to /usr/bin/make simply fills the latter's input queue. > > > The CPU has 24 threads and 64GB RAM, and lots of swap space, and those > > > values have worked well for some time. Now, though, I'm going to have to > > > limit the --jobs or the --load-average. > > > > > > On interrupting one such hang, I found that 32 install jobs had been > > > waiting to run; is this limit hard coded? It's certainly a suspicious number. > I take it the --load-average is what it says, an average, so it will jump > above the specified number if you have not limited the --jobs number. See above re. input queue. > > > I also saw "too many jobs" or something, and "could not read job > > > counter". > > > > > > Is it now bug-report time? > > You could set up a swap file, to avoid OOM situations, while you're tweaking > the --jobs & --load-average. The existing 64GiB swap partition is rarely touched, if ever. I've never seen an OOM error. I haven't touched jobs or loads for many months until today, nor have I seen a failure to read a job counter. Anyway, it still rankles that I can't use more than half the machine's power because of limits in portage. This can't be the only 64GiB machine in gentoo- land, surely. -- Regards, Peter.
Re: [gentoo-user] Too many simultaneous install jobs
On 6/20/24 11:29 AM, Peter Humphrey wrote: On Thursday, 20 June 2024 14:40:12 BST Michael wrote: On Thursday, 20 June 2024 14:27:18 BST Jack wrote: On 6/20/24 8:46 AM, Peter Humphrey wrote: While building a new KDE system (see my post a few minutes ago), I'm finding the system stalling because it can't handle all its install jobs. I have this set: $ grep '\-j' /etc/portage/make.conf EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" I don't know how much it would matter, but are you missing a number after --jobs? Without a number of jobs specified in make.conf emerge will not limit the number of packages it tries to build, except it will not start new jobs while there are at least --load-average=30 running already. MAKEOPTS="-j16 -l16" We went through all this at great length not long ago (months, perhaps: a certain A. McK had returned to the list for a while). /usr/bin/make will stop spawning make jobs once either (a) the number it's running reaches -j16 or (b) the load average of those reaches -l16. Portage sending more tasks to /usr/bin/make simply fills the latter's input queue. Again, I don't know if it matters in this case, but my understanding is that MAKEOPTS only affects jobs using make. I don't know if there are equivalent controls for ninja or other build systems. Might that be relevant here? If you run top, limit to running jobs and show the full command, does that give any hints? The CPU has 24 threads and 64GB RAM, and lots of swap space, and those values have worked well for some time. Now, though, I'm going to have to limit the --jobs or the --load-average. On interrupting one such hang, I found that 32 install jobs had been waiting to run; is this limit hard coded? It's certainly a suspicious number. I take it the --load-average is what it says, an average, so it will jump above the specified number if you have not limited the --jobs number. See above re. input queue. I also saw "too many jobs" or something, and "could not read job counter". Is it now bug-report time? You could set up a swap file, to avoid OOM situations, while you're tweaking the --jobs & --load-average. The existing 64GiB swap partition is rarely touched, if ever. I've never seen an OOM error. I haven't touched jobs or loads for many months until today, nor have I seen a failure to read a job counter. Anyway, it still rankles that I can't use more than half the machine's power because of limits in portage. This can't be the only 64GiB machine in gentoo- land, surely.
Re: [gentoo-user] Too many simultaneous install jobs
On 20/06/2024 16:29, Peter Humphrey wrote: Anyway, it still rankles that I can't use more than half the machine's power because of limits in portage. This can't be the only 64GiB machine in gentoo- land, surely. Well, I think my machine has 4x32GiB slots, and two are full, so that makes 64GiB here. They weren't even expensive - about £50 / DIMM iirc. Cheers, Wol
Re: [gentoo-user] Too many simultaneous install jobs
On Thursday, 20 June 2024 16:29:11 BST Peter Humphrey wrote: > On Thursday, 20 June 2024 14:40:12 BST Michael wrote: > > On Thursday, 20 June 2024 14:27:18 BST Jack wrote: > > > On 6/20/24 8:46 AM, Peter Humphrey wrote: > > > While building a new KDE system (see my post a few minutes ago), I'm > > > > > > > finding the system stalling because it can't handle all its install > > > > jobs. I have this set: > > > > > > > > $ grep '\-j' /etc/portage/make.conf > > > > EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" > > > > > > I don't know how much it would matter, but are you missing a number > > > after --jobs? > > > > Without a number of jobs specified in make.conf emerge will not limit the > > number of packages it tries to build, except it will not start new jobs > > while there are at least --load-average=30 running already. > > > > > > MAKEOPTS="-j16 -l16" > > We went through all this at great length not long ago (months, perhaps: a > certain A. McK had returned to the list for a while). /usr/bin/make will > stop spawning make jobs once either (a) the number it's running reaches > -j16 or (b) the load average of those reaches -l16. Portage sending more > tasks to /usr/bin/make simply fills the latter's input queue. Quite. Make will queue up anything above ~16 jobs, but emerge runs more than just make jobs. More and more emerge processes will kick off, up to ~30. Each emerge process will eventually launch make jobs, only for these to join a pile up in an ever congested make queue, unable to proceed further. At some point memory allocation and reallocation of queues appears to have become gnarly. Perhaps something in portage's python code leads to a race condition? I don't know if a combination of the queuing up of all these parent-child instructions and their parallelism can create an unchecked race condition, perhaps you reached some memory allocation limit, or indeed a bug in the code. Just loose suppositions of mine, not evidence by detailed debugging, let alone knowledge of python. > > > > The CPU has 24 threads and 64GB RAM, and lots of swap space, and those > > > > values have worked well for some time. Now, though, I'm going to have > > > > to > > > > limit the --jobs or the --load-average. > > > > > > > > On interrupting one such hang, I found that 32 install jobs had been > > > > waiting to run; is this limit hard coded? > > It's certainly a suspicious number. Apologies if I'm being dense here - why is it a suspicious number? I see a -- load-average of ~30 emerge instigated 'make install' jobs being queued up, while some previous 16 x make jobs are currently being processed. > > I take it the --load-average is what it says, an average, so it will jump > > above the specified number if you have not limited the --jobs number. > > See above re. input queue. > > > > > I also saw "too many jobs" or something, and "could not read job > > > > counter". > > > > > > > > Is it now bug-report time? > > > > You could set up a swap file, to avoid OOM situations, while you're > > tweaking the --jobs & --load-average. > > The existing 64GiB swap partition is rarely touched, if ever. I've never > seen an OOM error. I haven't touched jobs or loads for many months until > today, nor have I seen a failure to read a job counter. I don't know if counters are stored in memory, with running/completed/failed counts, or on disk. I can't think either DDR4, or an NVMe, would clog up their I/O channels, but you clearly witnessed a failure. Could this be a hardware glitch? You'll soon know if it shows up as a repeatable problem. > Anyway, it still rankles that I can't use more than half the machine's power > because of limits in portage. This can't be the only 64GiB machine in > gentoo- land, surely. I use 64G with no swap and MAKEOPTS="-j25 -l24.8" I haven't as yet come across a failure like yours, but I rarely try to run more than one emerge process at a time on this system. It's fast enough for my limited needs without having to increase the number of emerges at a time. On another PC which I often use as a binhost with 32G RAM, when I start two separate emerge processes manually with MAKEOPTS="-j10 -l9.8" I see swap being used a bit some times, especially when the PC user is hammering the browser for hours with many tabs open. Anyway, the MAKEOPTS directives control resource usage without hiccups. Does it make much of a difference in time saved running parallel emerges to require the addition of EMERGE_DEFAULT_OPTS? signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Too many simultaneous install jobs
On 6/20/24 8:46 AM, Peter Humphrey wrote: > Hello list, > > While building a new KDE system (see my post a few minutes ago), I'm finding > the system stalling because it can't handle all its install jobs. I have this > set: > > $ grep '\-j' /etc/portage/make.conf > EMERGE_DEFAULT_OPTS="--jobs --load-average=30 [...]" > MAKEOPTS="-j16 -l16" > > The CPU has 24 threads and 64GB RAM, and lots of swap space, and those values > have worked well for some time. Now, though, I'm going to have to limit the > --jobs or the --load-average. > > On interrupting one such hang, I found that 32 install jobs had been waiting > to run; is this limit hard coded? I also saw "too many jobs" or something, > and > "could not read job counter". > > Is it now bug-report time? It's not clear to me what "stalling" means here. Did portage stop doing any work as verified by ps or htop? Did it just spend a long time showing no progress? I do know what the 32 install jobs "waiting to run" is though. Or at least I'm pretty sure I know what it means. Recent portage has this change: https://gitweb.gentoo.org/proj/portage.git/commit/?id=825db01b91a37dcd9890ee5bf9f462ea524ac5cc "Add merge-wait FEATURES setting enabled by default" From the changelog: portage-3.0.62 (2024-02-22) -- * FEATURES: Add FEATURES="merge-wait", enabled by default, to control whether we do parallel merges of images to the live filesystem (bug #663324). If enabled, we serialize these merges. For now, this makes FEATURES="parallel-install" a no-op, but in future, it will be improved to allow parallel merges, just not while any packages are compiling. The purpose of this change is to solve cases where in one thread, portage is modifying /usr by deleting and installing files in a package such as "dev-python/setuptools-scm", while in another thread it is running a program that attempts to use those files and crashes because they are half-installed. When enabled (by default), portage will still compile lots of packages in parallel. But only when no packages are currently compiling, will they actually be merged to ROOT. Until then, they are in pending-merge state, and take up space in /var/tmp/portage. You can disable this FEATURES if you want. In https://bugs.gentoo.org/934382 portage is adding additional options: --jobs-merge-wait-threshold=X will cause portage to stop starting new jobs when X number of packages are in pending-merge state, and portage will copy the installed package files from the image to the root filesystem. Otherwise, portage will get there eventually but it might take a bit longer. -- Eli Schwartz OpenPGP_0x84818A6819AF4A9B.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Too many simultaneous install jobs
On 6/20/24 12:07 PM, Jack wrote:
> Again, I don't know if it matters in this case, but my understanding is
> that MAKEOPTS only affects jobs using make. I don't know if there are
> equivalent controls for ninja or other build systems. Might that be
> relevant here? If you run top, limit to running jobs and show the full
> command, does that give any hints?
It's an incomplete understanding.
ninja has the equivalent: NINJAOPTS. Other build systems can define
their own eclass variables too.
If a build system that doesn't use Make has no override, the ebuild will
extract the -j value *alone* from MAKEOPTS and use it. Potentially
load-average as well, if the other build system supports load averages.
Other make-compatible flags in MAKEOPTS will get ignored.
e.g
inherit multiprocessing
src_compile() {
my_build_system -j$(makeopts_jobs)
}
So, setting MAKEOPTS is usually sufficient to affect parallel
compilation in all ebuilds.
--
Eli Schwartz
OpenPGP_0x84818A6819AF4A9B.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
[gentoo-user] Bash, PS1, alias and how it works now.
Howdy, On my new rig, I'm trying to set up the PS1 and alias variables. I found the bash page on the Gentoo wiki but it seems to detail doing it for each user in their home directories. Since I'm the only one using this rig, I prefer to set it globally and have the commands behave the same way no matter who I am, root or dale. On my main rig still, I do this in /etc/bash/bashrc. It works just fine. On the new rig tho, it has a directory and things are broken up into different files for different things. Basically, the method has changed. It likely works better this way. Each package can have its own file. The wiki isn't helping since these files appear to be like scripts. I'm clueless on scripts. My one attempt to do it broke the whole thing. I had to emerge bash again with the config option to get the files back right. Is there a example file that gives a clueless scripts person a head start? I'd like to have it set the PS1 for root and user and have a alias section for several commands that applies to root and users. If there is a command to manage this sort of thing, I'm open to that too. I just want to set the PS1 and alias without breaking bash, or anything else. Also, I think the CPU thermal paste stuff has settled in a bit on the new rig. The temps have came down a little bit. Also, I found out that CPUTIN is a sensor that is likely placed similar to my older FX series CPU is. The new rig does run cooler if that is true. It should. CPU cooler is plenty large enough. Thanks for any help on the PS1 and bash settings. Dale :-) :-) P. S. Still trying to get around to pics and stuff. Fixed well pump, broke again. Fixed it again. Working so far. My back, not so much. :/
