Thibaut <ha...@slashdirt.org> [2023-06-02 11:09:48]:
Hi,
> the build is actually hung. dmesg might have more info.
So having following in buildbot log:
2023-06-01 23:53:12+0000 [-] command timed out: 3600 seconds without output
running [b'make', b'-j7', b'IGNORE_ERRORS=n m y', b'BUILD_LOG=1',
b'CONFIG_AUTOREMOVE=y', b'CONFIG_SIGNED_PACKAGES='], attempting to kill
2023-06-01 23:53:12+0000 [-] trying to kill process group 1528179
I've looked at the system logs around that time and found following:
Jun 01 22:23:19 audit[3844576]: AVC apparmor="DENIED" operation="mkdir"
info="Failed name lookup - name too long"
error=-36 profile="docker-default"
name="/shared-workdir/build/sdk/build_dir/hostpkg/gettext-0.21.1/gettext-tools/confdir3/confdir3/confdir3/confdir3...[snip
very long repeating pattern]...
confdir3/confdir3/confdir3/confdir3/confdir3" pid=3844576
comm="conftest" requested_mask="c" denied_mask="c" fsuid=1000 ouid=1000
Jun 01 22:23:45 kernel: conftest[3855174]: segfault at 0 ip 00007fe9581067e7
sp 00007ffd94ca2118 error 4 in libc-2.31.so[7fe958085000+159000]
Jun 01 22:23:45 kernel: Code: 00 00 00 48 39 d1 0f 82 47 67 06 00 0f 1f 80 00
00 00 00 48 89 f8 48 83 fa 10 0f 82 fa 00 00 00 48 83 fa 20 0f 87 3f 01 00 00
<0f> 10 06 0f 10 4c 16 f0 0f 11 07 0f 11 4c 17 f0 c3 0f 1f 84 00 00
Since the host is shared with other 3 build workers I can't be sure, that it
originated from that timeouted build.
> I’m happy to share my config if interested, it’s not a very complex setup.
That would be appreciated, thanks!
> I’m confused with that sentence: the du step shows 36G used, but df says all
> 60G are full; which suggests something *outside* of the build directory is
> eating space?
I just got following on openwrt-23.05 and osuosl-dock-02 build worker:
16G dl
36G shared-workdir
14G shared-workdir/build/sdk/staging_dir
13G shared-workdir/build/sdk/build_dir
8.6G shared-workdir/build/sdk/tmp/go-build
And looking at the other snapshot packages buildworkers, situation is similar
on some of them:
17G osuosl-dock-09/dl
17G osuosl-dock-10/dl
17G osuosl-dock-11/dl
17G osuosl-dock-12/dl
38G osuosl-dock-09/shared-workdir/build
35G osuosl-dock-10/shared-workdir/build
3.3G osuosl-dock-11/shared-workdir/build
7.1G osuosl-dock-12/shared-workdir/build
So probably some cruft accumulated in the shared-workdir for some reason, will
dig deeper once I find some time.
Cheers,
Petr
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
