On Sat, 2022-08-27 at 21:00 +0200, Alexander Kanavin wrote: > On Sat, 27 Aug 2022 at 20:45, Edwin Török <ed...@etorok.net> wrote: > > I don't think it is currently possible to limit the number of 'xz' > > processes though. > > This is a general problem and has been discussed many times over. > Bitbake historically doesn't limit the amount of compiler instances > or > any other processes that may start, and makes no promises about > avoiding OOM situations.
Indeed the more general problem of limiting memory used by compiler processes is more difficult to solve. > If you find yourself in one, you need to > tighten BB_NUMBER_THREADS/PARALLEL_MAKE until OOM goes away If XZ always uses 50% of available memory then adding more RAM won't help though. The only safe value would be BB_NUMBER_OF_THREADS=1. There seems to be an unintended consequence of XZ_MEMLIMIT=50%. AFAICT from the manpage 'xz -9' is meant to use 674MiB of memory at most. XZ_MEMLIMIT=50% is 32GiB in my case, so it appears to increase memory usage beyond what -9 would use (and compression time) by quite a lot. It can also create problems when decompressing on less powerful systems (xz says it might need up to 20% of the compressor's memory, so ~6.4 GiB in my case, but that number could be higher if compressed on a machine with even more RAM). Unless the intention here was to use as much memory as available to squeeze out a few extra savings in compressed image size I think that perhaps the default should be '50%' or '674MiB', whichever is smaller. (And then "add more RAM" is a perfectly valid way to get out of the OOM situation) What do you think, should I try a patch for that? Thanks, --Edwin > , or add > more RAM. More conservative defaults will never be conservative > enough > for every user, so we stick with defaults that make use of every > available CPU core. > > That said, there are recent bitbake patches to track 'memory > pressure' > and (I'm not sure exactly how) try to hold back if memory gets tight, > so you're welcome to take them and run experiments. > > Alex
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169969): https://lists.openembedded.org/g/openembedded-core/message/169969 Mute This Topic: https://lists.openembedded.org/mt/93294351/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
