On Fri, Jan 17, 2025 at 7:30 AM Adrian Bunk <b...@debian.org> wrote:
>
> On Thu, Jan 16, 2025 at 11:22:11PM +0100, Aurelien Jarno wrote:
> > On 2025-01-16 18:27, Aurelien Jarno wrote:
> > > Hi,
> > >
> > > On 2025-01-16 11:02, Simon McVittie wrote:
> > > > Control: found -1 2.58.93+dfsg-2
> > > >
> > > > On Thu, 16 Jan 2025 at 10:45:28 +0000, Simon McVittie wrote:
> > > > > librsvg/2.59.2+dfsg-1 failed to build on mips64el across multiple
> > > > > retries,
> > > > > most recently in
> > > > > <https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.59.2%2Bdfsg-1&stamp=1736998842&raw=0>.
> > > >
> > > > This does not seem to be a new problem. Looking back at older buildd
> > > > logs,
> > > > I can see the same error message
> > > >
> > > > > > error: failed to acquire jobserver token
> > > > > >
> > > > > > Caused by:
> > > > > > Bad address (os error 14)
> > > >
> > > > in most of the build logs since 2024-08-15:
> > > >
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-2&stamp=1723723132&raw=0
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-3&stamp=1723726896&raw=0
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-3&stamp=1723730758&raw=0
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-3&stamp=1723736907&raw=0
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-3&stamp=1723739367&raw=0
> > > > https://buildd.debian.org/status/fetch.php?pkg=librsvg&arch=mips64el&ver=2.58.93%2Bdfsg-3&stamp=1723746654&raw=0
> > > > etc.
> > > >
> > > > So I don't think this is genuinely a regression between testing and
> > > > unstable as it would appear to be from librsvg's inability to migrate.
> > > >
> > > > It seems to happen consistently on mipsel-osuosl-04 and mipsel-osuosl-05
> > > > (Quad Core Loongson 3A3000), with the package building successfully on
> > > > the
> > > > slower buildds -01 and -02 (Rhino Labs UTM8), and inconsistently
> > > > succeeding or failing on -03 (another Quad Core Loongson 3A3000).
> > > >
> > > > As a stopgap to allow librsvg to migrate, please could someone schedule
> > > > a build on mipsel-osuosl-01 or mipsel-osuosl-02 and see whether that
> > > > succeeds? It'll be more time-consuming but better than nothing.
> > >
> > > mipsel-osuosl-01 and mipsel-osuosl-02 have been disabled because they
> > > don't boot with the bullseye or bookworm kernels and thus we can't use
> > > them in unshare mode. Technically they have not yet been decommissioned,
> > > so while we can probably schedule a manual build in chroot mode, it
> > > looks wrong to get it uploaded in the archive.
> >
> > That said I have launched a build anyway on mipsel-osuosl-01 in case it
> > helps to learn something. It built successfully, so it points to a
> > hardware or kernel issue.
>
> There is recently a general instability/race in mips64el Rust/Go/Erlang
> builds, the most common pattern is that the build often (but not always)
> fails.
>
For Go, the following test fails frequently.
--- FAIL: TestRead (0.05s)
--- FAIL: TestRead/SpecialFile (0.05s)
read_test.go:25: read /sys/devices/system/cpu/online: bad address
read_test.go:25: read /sys/devices/system/cpu/online: bad address
I think the first time I saw that is
https://buildd.debian.org/status/fetch.php?pkg=golang-1.22&arch=mips64el&ver=1.22%7Erc1-2&stamp=1704593490&raw=0
Which uses kernel 6.1.69-1.
> Additionally, git also nearly always fails to build:
> https://buildd.debian.org/status/logs.php?pkg=git&arch=mips64el
>
> In the case of git, there is a clear pattern that the 5.10 -> 6.1
> upgrade causes the problem, and for the other issues I'd also
> suspect that.
>
> Looking at librsvg logs, 6.1.76-1 -> 6.1.99-1 seems where it broke.
>
> Looking through kernel changelogs, MIPS changes in this range were:
> - [mips*] scall: Save thread_info.syscall unconditionally on entry
> Closes: #1068365 (util-linux mkfds-multiplexing-{pselect6,poll,ppoll}
> failures)
> - [mips*] Clear Cause.BD in instruction_pointer_set
> - [mips*] reserve exception vector space ONLY ONCE
>
> OTOH, git already looks broken with 6.1.76-1.
>
> An interesting data point might be if any of these issues go away with
> the bookworm-backports kernel, librsvg/git/erlang seem to FTBFS pretty
> reliably with the (recent) 6.1 kernels.
>
> > Regards
> > Aurelien
>
> cu
> Adrian
>
--
Shengjing Zhu
