Hi! Definitely sounds like we have an issue we should deal with.
On Tue 16 May 2017 18:19, l...@gnu.org (Ludovic Courtès) writes: > here is the gcprof output I find gcprof useful when I want to improve runtime by removing allocation. However I don't find it useful when dealing with memory use issues; ymmv of course. > while compiling gnu/packages/python.scm, which defines 841 package > objects (structs) with 5 times more thunks of the form (lambda () > value): You mention later: > Also, for reference, loading python.go peaks at 315M RSS: > > $ \time ./pre-inst-env guile -c '(use-modules (gnu packages python))' > 0.18user 0.02system 0:00.18elapsed 112%CPU (0avgtext+0avgdata > 315648maxresident)k > 0inputs+0outputs (0major+7784minor)pagefaults 0swaps But this I don't understand. If I do a ./pre-inst-env guile and then load (gnu packages python), I get a 20MB heap size, a 35MB total private dirty memory and 52MB clean shared memory. (Measured using smaps via https://wingolog.org/pub/mem_usage.py). This is at commit 60c9e80444421c412ae3d0e7b4b224ef0e32947f. I just built the "time" package and I see similar numbers here. I can only think that the "time" package's numbers are bogus. > time(1) reports a maximum resident set size of 3.8G (though I see > something around 900MiB in ‘top’.) The only way Guile's memory usage can shrink in practice is if it recurses a lot on the stack and then returns those pages to the OS. I don't think libgc will return pages to the OS (though I could be wrong). So that would be a possibility to look into, if time can be trusted. > When compiling python.scm #:to 'cps, we end up with 1G max RSS in 6s. Measured with time? If this is the case it could be that python.scm is just a lot of code. Any compiler would take a lot if the IR size is 1 GB. > The only conclusion I can draw is that cps-to-bytecode compilation seems > to be responsible for most of the memory consumption. This is possible but I am not there yet. I don't see why compiling this file to CPS should cause memory usage of 1GB. That is 9000 memory words per textual line -- simply too much. Many unknowns here! Andy
