Hi Christopher, Thanks for your message! It’s great to see a summary of what’s been cooking in the Coordinator and your vision around it.
Christopher Baines <m...@cbaines.net> skribis: > More specifically, while the architecture is similar to daemon > offloading, there are some practical advantages. Since the switch to > Cuirass remote workers for ci.guix.gnu.org, the advantages of building > things across multiple machines in a coordinated manor have also become > clear. Additionally, there are things that the Guix Build Coordinator > enables, like automated retries, with the option to target specific > agents, which can help with avoiding issues with non-reproducible > failures, as well as helping to avoid issues when mixing QEMU emulated > and native builds. These are nice examples of current QA blind spots that would be nice to address. [...] > This is a topic I haven't got directly involved in, until now. I'm just > a volunteer, in some ways the most involved I am is that I host an idle > ARM build machine, I don't have any particular connection in the default > approach to substitutes for users, or the hardware currently > involved. It would be great to have that OverDrive plugged in behind ci.guix because we’re still short on CPU power for ARM. (Not great we had forgotten about this one!) It could be reconfigured with a config similar to <https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/overdrive1.scm>, with a different host name and IP. We can discuss the details in a separate thread and/or on guix-sysadmin, let me know! > However, I think some of the stuff I've been working on could be > helpful, as I say, I think the Guix Build Coordinator is a step > forward in terms of building things for substitutes, and that shows in > the substitute availability percentages. > > Is there still a path to bring some of these benefits to users, and if > so, what things need doing? My understanding is that, to build substitutes, we need more than the Coordinator; we also need something to perform evaluations, similar to the “top half” of Cuirass and the Data Service, right? How would you envision setting it up? To me, the way forward is not necessarily substitutes; it could be QA, as you showed that it has helpful features. You already have access to bayfront and it’d be great to experiment there. Do you have ideas on how to do make progress on that front? Besides, I feel that the Data Service has a huge role to play with a variety of applications, including QA, but also way beyond as you showed in the blog post¹. I think it’s time take advantage of it. :-) Specifically, as far as QA is concerned, I think we should place the Data Service at the center of the stage. It already does things that Cuirass cannot and will not do, notably: linting, challenging substitute servers to estimate reproducibility, and recording the history of all this. Couldn’t it gather these other QA metrics mentioned above, possibly from a Coordinator-powered bayfront? What I’d love to see is a set of “skins” for the Data Service: purpose-specific interfaces to the service. There’s already the reproducibility one, the one for package versions, etc. It would be great to have a qa.guix.gnu.org entry point with a simplified interface that would only show (say) reproducibility data, lint warnings, and so forth. (Then we can also think of other skins, like Guix Weekly News, a page dedicated to browsing package version history, a security-related page, etc. But now I feel like a spoiled kid asking for yet more presents. :-)) How does that sound? Thanks! Ludo’. ¹ https://guix.gnu.org/en/blog/2020/introduction-to-the-guix-data-service-the-missing-blog-post/
