On Friday, 25 October 2019 06:31:03 BST Raffaele BELARDI wrote: > 8-core CPU: > > EMERGE_DEFAULT_OPTS="--quiet-build --keep-going --jobs 9 --load-average 9" > > MAKEOPTS="-j9 -l9" > > Works fine except when both Firefox and Thunderbird need update, in that > case emerge typically tries to build them in parallel and one gets OOM > killed due to insufficient swap space (1G swap, 16G RAM). I will increase > the swap but I'd like to know: Is there a way to tell emerge to normally > run 9 parallel jobs but limit to 1 when it is building one of the two > monsters? > > Thanks, > > raffaele
Good advice has already been given, but here are some supporting thoughts based on my experience of trying to emerge mammoth builds on old systems with low RAM. I can't recall how much memory each thread of a FF or TB compile chews up, but if you allow for up to 2G per thread, with -j9 your 16G RAM *will* be exhausted at some point. Swapping will commence then in earnest and a I/O offloading of RAM into swap and back again as threads gasp for adequate memory will be sustained, while demand for more RAM than available is present. This creates a bottleneck which will prolong the emerge duration. With enough swap you won't suffer any more OOMs, but you could find emerge times increase exponentially. Therefore reducing the total number of jobs for these two packages will help in both cases. Setting up an env directive as already advised in previous responses to restrict both packages to fewer jobs will work, but in some cases it can be suboptimal. When only one package needs updating, why should it be emerged at a lesser speed than your system can support? Even when both packages are emerged at the same time, the memory crunch may only take place for a limited period, at a time when both packages will be using maximum memory per thread. To make matters worse ebuilds and source code changes, compilers change and trying to optimise your emerge soon becomes a moving beast, so some experimentation will be necessary. I think reducing your job/load count to 1 would be excessive. Reducing it down to 5 ought to be adequate, but I suggest to increase your swap. You could just use a swap file for this purpose. PS. In an ideal AI world, portage would know how much memory is necessary for a given package and would auto-adjust the number of jobs to minimise swapping given any amount of RAM. In an even more ideal world, it would be able to do this in real time. :-) -- Regards, Mick
signature.asc
Description: This is a digitally signed message part.
