Hi! While this sounds like a great idea, should this be a GCD (already)?
I'd assume that as long as hardware/funding and sysadmins are found, no one would be opposed to having non-EU binary substitute servers. If Guix has disposable funds, a GCD of the form "let's use some of those to set up servers" would make sense to me, or if servers had been secured "let's use those servers as a binary substitute server" would make sense, but imo in its current form this GCD lacks actionability: there's no clarity as to who would do what if this were to be accepted Kind regards, pinoaffe Antero Mejr <m...@antr.me> writes: > I am looking for feedback/sponsors for a new GCD proposal to set up one > or more international (outside EU) binary substitute servers in order to > improve download speeds. > > The PR is not ready yet because the proposal needs to be a more concrete > plan. I would like to collect people's feedback before updating it - if > you all think this issue is important, whether colocation vs. cloud > would be preferable, where the servers should be located, or any other > comments/concerns. > > The working repository is here: > https://codeberg.org/antero/guix-consensus-documents/src/branch/wip-substitutes > > Here is the initial markdown text: > > title: International substitute servers > id: 6 > status: draft > discussion: <draft|submitted|accepted|withdrawn|deprecated> > authors: Antero Mejr > sponsors: <Sponsor Name> > date: <date when the discussion period starts> > draft-date: 2025-08-07 > discussion-date: <date when the discussion period starts> > deliberation-date: <date when the deliberation starts> > SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-or-later > --- > > # Summary > > The official binary substitute servers for Guix, `ci.guix.gnu.org` and > `bordeaux`, are both located in Western Europe. Users outside Western Europe > may > experience slower speeds when downloading substitutes: 50-200 kbps is not > uncommon. Since substitutes can be quite large, this causes significant slow > down when performing package operations. This RFC proposes the establishment > of > additional substitute servers in other regions. > > # Motivation > > In the present state, users outside Europe experience substitute download > speeds > on the order of kilobits per second. This comprises some of the perceived > "slowness" of Guix, as mentioned in all three of the [Guix > surveys](https://guix.gnu.org/en/blog/2025/guix-user-and-contributor-survey-2024-the-results-part-1/). > Other package managers with distributed substitute servers achieve download > speeds of tens or even hundreds of megabytes per second, many times faster > than > the single Guix substitute server can provide. > > Common dependency packages such as LLVM can be large, around 50-200 MB. > Downloading substitutes for them on such a slow connection will take at least > 5 > minutes, and that is only for a single dependency. This is a significant > impediment to package management operations and workflows. > > Distributing substitutes can also increase reliability. As it stands, if > `ci.guix.gnu.org` is not available, the user will not be able to download > substitutes at all. It also assists with reproducibility: more identical > substitute builds means a higher degree of assurance of deterministic, > reproducible packages. > > # Detailed Design > > Many package managers make available dozens of substitute servers (often > called > "mirrors") from which to download compiled packages. Usually package managers > have some sort of configuration option to select the mirror with the fastest > connection, typically the mirror that is geographically closest, and possibly > some sort of auto-detection of connection speeds. Guix does not currently have > such a configuration option, and defaults to `ci.guix.gnu.org`. The addition > of > a mirror selection configuration option is part of this proposal. Such an > option > could be a `guix-configuration` option of `guix-service-type`. > > The Guix installer, for both the ISO and foreign-distro forms, would have an > additional prompt to select a substitute server from a list. > > Since many Guix users and developers are based in the United States, a > US-based > server would be a sensible first step beyond the European region. From there > community feedback can be solicited to determine locations for future servers, > so long as sufficient funds are available. > > ## Cost of Reverting > > Cost scale: 0 - No incompatibility > > Users of Guix System would experience no change, as substitute server > authentication is handled automatically. Users of Guix on foreign distros who > are installing for the first time would experience no change, so long as they > choose to authorize binary substitutes at install time. > > For users of Guix that is already installed on a foreign distro, they may need > to follow [these > steps](https://guix.gnu.org/manual/en/html_node/Substitute-Server-Authorization.html) > to authorize the new substitute servers if they wish to use them. Otherwise > they > would not see the benefits. > > No deprecation of existing substitute servers would be necessary. > > Depending on whether colocation is used, maintenance of the substitute server > may require a Guix developer in the geographic area of the server, with proper > vetting/access clearance to perform administrative tasks directly on the > hardware. Remote maintenance of the server(s) would be possible for most > operations. > > Should a new substitute server need to be decommissioned (for whatever > reason), > there would need to be a process established for doing so without breaking > compatibility. This would involve having the decommissioned substitute server > URL redirect to a different active and authorized substitute server. > > # Drawbacks and Open Issues > > The main drawback of maintaining additional substitute servers is cost, in > both > the initial investment and recurring costs. For example, Nix serves their > infrastructure through [Amazon > AWS](https://discourse.nixos.org/t/s3-sponsorship-extension-more-resources-to-build-a-more-sustainable-nix/50936) > and their recurring costs are $15,000 per month. > > However, Guix could potentially mitigate some of the costs. Making use of > donated hardware, and using colocation instead of the cloud could reduce the > monthly spend. Also the FSF may have resources that can provide servers or > colocation services at a reduced price. Guix also has an FSF [Working Together > Fund](https://my.fsf.org/civicrm/contribute/transact?reset=1&id=50) that could > potentially be used to fund substitute servers. > > The [Guix Foundation](https://foundation.guix.info/) may also be interested in > providing financial support for international substitute servers.
