Hi Emilio, On 2026-06-24 14:06, Emilio Pozuelo Monfort wrote: > Hi Aurelien, > > On 27/05/2026 21:31, Aurelien Jarno wrote: > > Hi Paul and release team, > > > > On 2026-05-26 21:30, Paul Gevers wrote: > > > Hi riscv64 porters, > > > > > > In bug 1092153 [1] we discussed the slowness of the riscv64 hardware > > > available for Debian. At the time we accepted the situation as a > > > bootstrapping problem. Recently we've witnessed some delays because of > > > riscv64 build speed (e.g. migration delay for linux kernel CVE fixes), and > > > since riscv64 got added to ci.debian.net it needed shepherding to stay up > > > and a lot of tests actually fail because of the slowness. > > > > > > Hence, I'd like to request an update on the situation. When do you think > > > the > > > situation can be improved realistically? As an example, I've heard that > > > for > > > some of the faster hardware, the kernel patches still aren't accepted > > > upstream. Also, on what timescale can the slow hardware (both buildds and > > > the ci.d.n hardware) be replaced with faster hardware that runs reliably? > > > > I am answering here about the build daemons part as the debci part has > > already been answered by Bo Yu. > > > > First of all the requirements for the build daemons is a bit more > > challenging than debci, as we want to run them on our own hardware, > > using the Debian (bpo) kernel, in a rack-mountable format. As pointed > > out in bug #1092153, we had identified two possible hardware platforms. > > Unfortunately neither turn out to be suitable for build daemons. > > Complete kernel support for the HiFive Premier P550 is still lacking, > > and we experienced some stability issues with the Milk-V Pioneer. > > > > We therefore looked for alternatives and found the Milk-V Jupiter board > > with a SpacemiT K1 CPU to be a good candidate, with a lot of upstreaming > > work happening. The Debian 7.0 backports kernel is now suitable to run a > > build daemon. Testing shows a performance improvement by a factor of 1.7 > > to 2.2, depending on the level of parallelism. For example, the Linux > > kernel can be built in about 9 hours and rustc in about 14 hours. We > > also expect an additional 10% gain once cpufreq support is available, > > as the CPU is currently not running at its nominal speed. In addition > > support for vector instructions may further improve performance for some > > specific packages. > > > > The other option that appeared recently is the SpacemiT K3. The > > upstreaming process is progressing quite fast, also benefiting from the > > existing K1 code. The Debian backports kernel is already able to boot on > > such a machine, but with a quite limited set of devices. Kernel 7.2 or > > 7.3 (the 7.2 merge window is not yet fully closed) should provide > > sufficient support for use as a build daemons, so likely around fall. > > Online benchmarks indicate we should expect to halve the build time > > compared to the K1, although we have not yet been able to validate this > > ourselves due to lack of hardware. > > > > Regarding the timescale, we started the process of upgrading the build > > daemons with the Spacemit K1, beginning with MANDA. The first two boards > > arrived there last week and are waiting to be installed. The plan is to > > upgrade the two other boards once we are sure everything is working fine > > with the two first boards. > > Any progress on those first boards? Have they been installed, and has > testing on them begun?
Unfortunately they haven't been installed yet. There is an option to send someone on site instead, but that cannot happen before around mid July. > > For OSUOSL, we still have to plan and decide if we go with the K1 or if > > we wait for the K3. That will depends on how things move in the next > > weeks on the K3 side (upstreaming, access to K3 hardware, etc.). > > My preference would be to have some K3 boards, even if that means waiting a > bit longer, so that we can further cut build time for large packages. I have been able to progress a bit on that front, thanks to SpacemiT, I got access to a K3 board. On the good side, the performance are impressive compared to the Unmatched boards we currently use, both in terms of CPU and NVMe speed. I measured a speed improvement of up between 6 to 8 compared to the current buildds. For instance the linux package from experimental builds in 2h30. On the bad side upstream kernel support is not fully there, although I am currently running a 7.1 kernel with ~50 patches, that translates to ~25 patches against the future 7.2 kernel. Let's hope most of them can be merged for 7.3. There are also a few additional issues that I have found, and which will need to be fixed before we can consider using SpacemiT K3 based hardware as buildd. I have already sent two fixes [1] [2], but there is also at least reboot support and a system crash with vector code that need to be fixed. Regards Aurelien [1] https://lore.kernel.org/linux-riscv/[email protected]/ [2] https://lore.kernel.org/linux-riscv/[email protected] -- Aurelien Jarno GPG: 4096R/1DDD8C9B [email protected] http://aurel32.net
