Hi Christopher, Apologies for not answering earlier.
Christopher Baines <m...@cbaines.net> skribis: > One of the memory problems I'm having relates to the Guix inferior > processes that the data service uses when computing derivations. The > data serivce goes through the list of systems (x86_64-linux, > aarch64-linux, ...) and because the data cached for x86_64-linux > probably doesn't relate to aarch64-linux, there's some code that > attempts to clear the caches [1]. > > 1: > https://git.savannah.gnu.org/cgit/guix/data-service.git/tree/guix-data-service/jobs/load-new-guix-revision.scm#n1970 > > Unfortunately this code has to reach in to Guix internals to try and do > this, and it does reduce the heap usage significantly, but this doesn't > result in stable memory usage. Each system processed seems to add about > 250MiB of data to the Guile heap that isn't cleared out. To me that > sounds like a lot of memory, but there's also a lot of systems/targets, > so overall this leads to the inferior process using with around 6GiB of > data in the heap after processing all the systems/targets. This peak > memory usage really limits how much the machine can do. Did you consider running one inferior per system type, as a way to sidestep the problem? This is what ‘cuirass evaluate’ does and it manages to run typically 4 of them in parallel (x86_64-linux, i686-linux, aarch64-linux, and powerpc64le-linux). Memory usage may be high but at least it’s not a blocker. https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/src/cuirass/scripts/evaluate.scm#n137 Ludo’.
