Ludovic Courtès writes: > Hi Roel, > > Roel Janssen <r...@gnu.org> skribis: > >> Ludovic Courtès writes: > > [...] > >>>> FWIW, on our NFS-mounted /gnu, the 'guix environment' command takes at >>>> least 20 seconds, but for any reasonably big environment it takes more >>>> than one minute. The biggest bottleneck here is the disk latency. >>>> Could it be that 'guix environment' accesses many files? If we could >>>> reduce that, it would speed things up (at least for us ;)). >>> >>> Interesting. Does guix-daemon access /gnu over NFS too? >> >> Yes. > > That is going to make builds verrrry slow. I would suggest having > guix-daemon operate on the local /gnu (that’s what Ricardo did, IIRC).
That definitely speeds things up about 50%. So, running the environment command from a node that is connected to the node that runs guix-daemon, will take about 30 seconds instead of one minute. This could be made faster by letting 'guix-daemon' do more of the work, and let the client-side 'guix' only do the minimum. (I know.. this remote-guix-daemon stuff voids the warranty.. ;)) The NFS overhead is pretty large. Maybe we can better tune it, and if so, document how to tune it for GNU Guix. I already talked to our storage expert, and tuning boils down to using fast disks, a large-enough NFS cache, and low-latency network equipment. The reason we have /gnu NFS-mounted on the build node is that we can then very easily replace the node when it would have a hardware failure, without even losing the ability to run programs that were already installed. > >>> We stat a lot mostly to access all the Guix modules. Are they on NFS >>> too? >> >> Yes. > > OK. > >> Here's my output: >> >> [rjanssen2@hpcguix ~]$ strace -c guix environment --ad-hoc coreutils --pure >> -- true >> % time seconds usecs/call calls errors syscall >> ------ ----------- ----------- --------- --------- ---------------- >> 41.79 0.038363 22 1782 188 open >> 14.71 0.013500 3 4769 1747 stat >> 13.19 0.012106 2 8033 read >> 6.06 0.005561 1854 3 1 wait4 >> 5.83 0.005356 8 633 147 futex >> 4.15 0.003814 0 10020 write >> 3.25 0.002987 157 19 clone > > [...] > >> ------ ----------- ----------- --------- --------- ---------------- >> 100.00 0.091800 36339 2095 total >> >> >> So the main difference is that calls to 'open' take a lot longer. >> There's little we can do, other than come up with a good caching >> mechanism. > > We should make sure these 1782 calls are all warranted (e.g., make sure > we don’t open the same file twice; this could be easily diagnosed by > looking at the output of ‘strace’.) > > But then, as you say, it’s a process that opens lots of files. > > We could cache package lookups, and maybe that’d help a little bit. > Currently we use ‘fold-packages’, which loads every single package > module, and then we look for a package named “coreutils” among them; > with a cache, we could directly load (gnu packages base) and its > dependencies. > > Ludo’. Running 'guix package -A' only takes a couple of seconds. So I don't think that loading the Scheme modules is really a big issue. Kind regards, Roel Janssen
