Hi Guix, (I’m CC’ing swedebugia and Jelle on this, since I promised both of them off-list that I would publish this work eventually.)
Most of us know that the JavaScript situation with Guix is a little dire. Particularly, some JavaScript packages have circular dependencies and most are not bootstrappable. In practice, this has led to Guix having almost no JavaScript packages, making it not suitable for JavaScript-heavy Web development. It also means that packaging high-profile, Web-based programs like GitLab or Mastodon is so difficult as to be impossible. Given these obstacles, I thought it would be interesting to explore how far you can get if you stop worrying about bootstrapping the packages, and just use the pre-built ones. It turns out that you can go as far as you like this way. In fact, I’ve built a recursive importer and build-system for NPM that can import and “build” large and complicated packages such as Babel. It does this by downloading the pre-built tarballs from NPM, and then installing them using the “npm” command with some dependency rewriting. It’s actually pretty simple. The main trick was finding the right flags to pass to NPM, and figuring out how to trick it into pulling dependencies from the store rather than the Web. The code is at <https://git.ngyro.com/guix-npm>. I know that this is not great for Guix proper, but I thought that this might be a good use-case for channels. In a channel we could build out the JavaScript ecosystem in Guix on top of a bunch of “binary” packages, and then try to reduce this set of “binary” packages gradually. I think that having a useful JavaScript ecosystem, even if it is problematic, might help motivate folks to work out the problems. For example, it would be a lot more exciting to work on bootstrapping TypeScript if it were the only thing standing in the way of getting a Mastodon service in Guix. That’s the high-level overview. The rest of this message is the technical details of how to try out the importer, build packages, etc. Here’s how to make “@babel/cli” work with a Babel plugin called “transform-arrow-functions”. Clone the repo and build it like you would build Guix normally, except for where you would usually use “guix environment guix”. Instead, use guix environment guix --ad-hoc -l guile-semver.scm to ensure that the Semantic Versioning library required by the importer is available. After this, continue as usual. Once built, create a new file at “gnu/packages/node-babel.scm” with the following contents: (define-module (gnu packages node-babel) #:use-module (guix build-system other-node) #:use-module (guix download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages)) Then, you can run ./pre-inst-env guix import npm-binary -r @babel/cli \ | tee -a gnu/packages/node-babel.scm (using “tee” is nice ’cause you can watch the output as it goes – it has a lot of dependencies, so it takes a little while). Get the plugin, too: ./pre-inst-env guix import npm-binary -r \ @babel/plugin-transform-arrow-functions \ | tee -a gnu/packages/node-babel.scm There’s a package in there that is macOS specific, so it has to be removed. It’s called “fsevents”, and it is dependency of “chokidar”. Everything works fine if it is simply deleted. At this point, you can launch an environment with the packages (be sure to include “node”, as that reminds Guix to set “NODE_PATH”): ./pre-inst-env guix environment --ad-hoc node node-babel-cli--fiio \ node-babel-plugin-transform-arrow-functions--fiio (For the curious, “fiio” is an initialism for a phrase used in Christopher Lemmer Webber’s article about JavaScript packaging: <http://dustycloud.org/blog/javascript-packaging-dystopia/>. It serves as a reminder that these are not nice packages.) Unfortunately, Babel only looks for plugins relative to the current directory, so you’ll need a symlink: “ln -s $NODE_PATH”. With all this in place, it’s time for a test! echo '() => { return true; };' > test.js babel --plugins=@babel/plugin-transform-arrow-functions test.js This prints: (function () { return true; }); which is good, since it is transforming the modern lambda syntax with the arrow “=>” into the old syntax with the “function” keyword. Hooray! Now for some more notes. I’ve tried a handful of packages, and have had good results so far. Sometimes things need a little fiddling (like “fsevents”), but the results are usable. Both the importer and the build system are mostly new code. For the importer, the main difference is that it uses the NPM version constraints to make sure the recursively imported packages have the expected versions. It does this via a Guile library that I wrote. One limitation is that the recursive importer does not check versions for existing Guix packages. If there is a package with the right name, it is accepted even if it doesn’t satisfy the version constraints. Guix’s “recursive-import” function would have to be changed to fix this. The build system uses “npm” for everything, rather than trying to emulate “npm” in Scheme (like the recently added build system does). I’m not sure if this is better or worse, really, but it works nicely for this particular use-case. It works in theory to build packages from source, too. One test I did was importing all of jQuery’s development dependencies as binary packages, and then building jQuery from source. I’ve come to think that bootstrapping JavaScript might be easier than it looks. As time goes on, Node gets better at the newer JavaScript features. This removes the need for things like Babel or Rollup, since with some care, Node can run the source directly with out any transformations or bundling. That being said, TypeScript looks to be a major issue, as it is used in many fundamental JavaScript packages and it is not bootstrappable. I’m not sure in what capacity I want to pursue this. It’s been sitting dormant on my computer for while, so I thought sharing it would be better than letting it fall by the wayside. I hope it proves useful one way or another. If you got this far, thanks for reading! :) -- Tim
