Hello Guix! A heads-up to share the cognitive burden related to this topic. :-)
So, we have two problems: compilation time, and memory consumption. I *think* I’ve identified one of the major causes for both in Guile, though it’s too early to say exactly how much this will impact resource consumption for a full Guix compilation. See <https://lists.gnu.org/archive/html/guile-devel/2017-09/msg00031.html> for details. When that is fixed, we’ll still have a performance problem: building all of Guix will still take more time than we’d like, and it won’t get better as we add new files. So we need to address this. This has been discussed informally many times, and here’s a summary of the ideas I’m aware of: 1. Build Guix as separate derivations: the first derivation builds the closure of (guix packages), the second one builds the closure of (guix scripts *), and finally we build (gnu *). We could also have derivations for the non-Scheme parts: the daemon, the manual, etc. (That amounts to compartmentalizing Guix in sub-packages, which in a way solves the bootstrapping issue that Pjotr complained about at some point—the fact that to build Guix one needs Guile, Guile-JSON, GnuTLS, Autotools, etc.) The advantage is that when running ‘guix pull’ frequently, you won’t have to recompile all of these. However, you’ll almost always have to rebuild (gnu packages *), which is the longest part. 2. Build all of Guix like the ‘guix’ package does, and hope that we can get a substitute. Bootstrapping issue: to do that, we first need compute the derivation of this new ‘guix’ package. Thus, we at least need to build the closure of (guix packages), which should take a minute or so, after which we can compute the derivation, which could take a couple of minutes maybe. The problem is that building all of Guix (including running the test suite) takes some time, potentially more than the interval between two subsequent pushes to the repo. Thus, it’s quite likely that the build farm would always be lagging behind. We could work around that by having the build farm automatically tag commits for which it has successfully built Guix. That would introduce delays in deploying the latest Guix to users, but maybe that can be short enough to be acceptable. 2b. To address the bootstrapping issue above, we also discussed the possibility of setting up a “meta-derivation” service: you’d give it a Git commit, and it’d return the derivation of ‘guix’ for that commit. Less computation would take place on the user side, but that doesn’t reduce the delay mentioned above. It also has the downside of introducing another service without which using Guix is more painful. I think that’s about it. Thoughts? Hacks? :-) Ludo’.
