Source: netplan.io
Version: 0.103-3
Severity: important
Dear maintainer,
Somewhere between 2021-11-06 and 2021-11-12 the autopkgtest of
netplan.io has regressed on the ci.debian.net infrastructure on
ppc64el. It started to tmpfail after about 15 to 20 minutes. As
tmpfails are immediately retried by the Debian migration software,
this is causing quite some load. Do you know what goes wrong? Can we
help identify how this is possible?
I'm putting netplan.io on the reject_list for ppc64el for now. Please
reach out if you need our help to investigate.
I have diffed the artifacts for you in the attachment between the last
test that passed and the first that tmpfailed.
The end of the log looks like:
autopkgtest [20:28:47]: test cloud-init: ./debian/tests/prepare-testbed.sh &&
./debian/tests/cloud-init.sh
autopkgtest [20:28:47]: test cloud-init: [-----------------------
+ rm -f /etc/network/interfaces
+ systemctl unmask systemd-networkd.service
+ systemctl unmask systemd-networkd.socket
+ systemctl unmask systemd-networkd-wait-online.service
+ systemctl enable --now systemd-networkd.service
Created symlink /etc/systemd/system/dbus-org.freedesktop.network1.service →
/lib/systemd/system/systemd-networkd.service.
Created symlink
/etc/systemd/system/multi-user.target.wants/systemd-networkd.service →
/lib/systemd/system/systemd-networkd.service.
Created symlink
/etc/systemd/system/sockets.target.wants/systemd-networkd.socket →
/lib/systemd/system/systemd-networkd.socket.
Created symlink
/etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service
→ /lib/systemd/system/systemd-networkd-wait-online.service.
+ systemctl unmask systemd-resolved.service
+ systemctl enable --now systemd-resolved.service
Created symlink /etc/systemd/system/dbus-org.freedesktop.resolve1.service →
/lib/systemd/system/systemd-resolved.service.
Created symlink
/etc/systemd/system/multi-user.target.wants/systemd-resolved.service →
/lib/systemd/system/systemd-resolved.service.
+ mount -o remount,rw /sys
+ systemctl unmask systemd-udevd.service
Removed /etc/systemd/system/systemd-udevd.service.
+ systemctl start systemd-udevd.service
INFO: Preparing configuration
● systemd-networkd.service - Network Configuration
Loaded: loaded (/lib/systemd/system/systemd-networkd.service; enabled;
vendor preset: enabled)
Active: active (running) since Wed 2021-11-17 20:28:48 UTC; 770ms ago
TriggeredBy: ● systemd-networkd.socket
Docs: man:systemd-networkd.service(8)
Main PID: 1310 (systemd-network)
Status: "Processing requests..."
Tasks: 1 (limit: 9657)
Memory: 4.5M
CPU: 60ms
CGroup: /system.slice/systemd-networkd.service
└─1310 /lib/systemd/systemd-networkd
Nov 17 20:28:48 ci-321-dd1f0929 systemd[1]: Starting Network Configuration...
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Link UP
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Gained carrier
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: lo: Link UP
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: lo: Gained carrier
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Gained IPv6LL
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: Enumeration completed
Nov 17 20:28:48 ci-321-dd1f0929 systemd[1]: Started Network Configuration.
Removed
/etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service.
Removed /etc/systemd/system/sockets.target.wants/systemd-networkd.socket.
Removed /etc/systemd/system/multi-user.target.wants/systemd-networkd.service.
Removed /etc/systemd/system/dbus-org.freedesktop.network1.service.
Created symlink
/etc/systemd/system/basic.target.requires/cloud-init-dummy.service →
/etc/systemd/system/cloud-init-dummy.service.
INFO: Configuration written, rebooting...
bash: line 1: 1243 Killed bash -ec
'./debian/tests/prepare-testbed.sh && ./debian/tests/cloud-init.sh' 2> >(tee -a
/tmp/autopkgtest-lxc.qs43ksy_/downtmp/cloud-init-stderr >&2) > >(tee -a
/tmp/autopkgtest-lxc.qs43ksy_/downtmp/cloud-init-stdout)
autopkgtest [20:28:49]: test process requested reboot with marker config
<VirtSubproc>: failure: timed out waiting for container ci-321-dd1f0929 to
start; last runlevel "unknown
"
autopkgtest [20:31:00]: ERROR: testbed failure: cannot send to testbed: [Errno
32] Broken pipe
Paul
https://ci.debian.net/packages/n/netplan.io/testing/ppc64el/
diff -r pass/autostart-packages fail/autostart-packages
1c1
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
5,6c5,6
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
10c10
< python3 3.9.2-3
---
> python3 3.9.7-1
13,14c13,14
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/bonds-packages fail/bonds-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/bridges-packages fail/bridges-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/cloud-init-packages fail/cloud-init-packages
1c1
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
5,6c5,6
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
10c10
< python3 3.9.2-3
---
> python3 3.9.7-1
13,14c13,14
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/duration fail/duration
1c1
< 11745
---
> 1001
diff -r pass/ethernets-packages fail/ethernets-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/exitcode fail/exitcode
1c1
< 2
---
> 16
diff -r pass/ovs-packages fail/ovs-packages
11c11
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
30,31c30,31
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
44c44
< python3 3.9.2-3
---
> python3 3.9.7-1
49,50c49,50
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/regressions-packages fail/regressions-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/routing-packages fail/routing-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/scenarios-packages fail/scenarios-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/summary fail/summary
1,2c1,2
< ovs FLAKY timed out
< ethernets PASS
---
> ovs FLAKY non-zero exit status 1
> ethernets FAIL non-zero exit status 1
4,10c4,10
< bonds PASS
< routing PASS
< vlans PASS
< wifi PASS
< tunnels PASS
< scenarios PASS
< regressions PASS
---
> bonds FAIL non-zero exit status 1
> routing FAIL non-zero exit status 1
> vlans FLAKY non-zero exit status 1
> wifi FLAKY non-zero exit status 1
> tunnels FAIL non-zero exit status 1
> scenarios FLAKY non-zero exit status 1
> regressions FAIL non-zero exit status 1
12c12
< cloud-init PASS
---
> testbed failure: cannot send to testbed: [Errno 32] Broken pipe
diff -r pass/testbed-packages fail/testbed-packages
6c6
< bash 5.1-3+b2
---
> bash 5.1-3.1
13c13
< coreutils 8.32-4
---
> coreutils 8.32-4.1
51c51
< iproute2 5.15.0-1
---
> iproute2 5.14.0-1
126c126
< libpam-systemd 249.5-2
---
> libpam-systemd 249.6-1
132c132
< libpython3.9-minimal 3.9.7-4
---
> libpython3.9-minimal 3.9.8-1
139,140c139,140
< libsemanage-common 3.1-1
< libsemanage1 3.1-1+b2
---
> libsemanage-common 3.1-2
> libsemanage1 3.1-2
147c147
< libsystemd0 249.5-2
---
> libsystemd0 249.6-1
152c152
< libudev1 249.5-2
---
> libudev1 249.6-1
160c160
< login 1:4.8.1-1
---
> login 1:4.8.1-1.1
170,172c170,172
< openssh-client 1:8.4p1-6
< openssh-server 1:8.4p1-6
< openssh-sftp-server 1:8.4p1-6
---
> openssh-client 1:8.7p1-1
> openssh-server 1:8.7p1-1
> openssh-sftp-server 1:8.7p1-1
174c174
< passwd 1:4.8.1-1
---
> passwd 1:4.8.1-1.1
181,182c181,182
< python3-minimal 3.9.2-3
< python3.9-minimal 3.9.7-4
---
> python3-minimal 3.9.7-1
> python3.9-minimal 3.9.8-1
187,188c187,188
< systemd 249.5-2
< systemd-sysv 249.5-2
---
> systemd 249.6-1
> systemd-sysv 249.6-1
diff -r pass/testinfo.json fail/testinfo.json
2c2
< "virt_server": "autopkgtest-virt-lxc --sudo --name ci-310-f38c586c
autopkgtest-testing-ppc64el",
---
> "virt_server": "autopkgtest-virt-lxc --sudo --name ci-316-152ee3eb
> autopkgtest-testing-ppc64el",
diff -r pass/trigger fail/trigger
1c1
< iproute2%2F5.15.0-1
---
> systemd%2F249.6-1
diff -r pass/tunnels-packages fail/tunnels-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/vlans-packages fail/vlans-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/wifi-packages fail/wifi-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/worker fail/worker
1c1
< ci-worker-ppc64el-02
---
> ci-worker-ppc64el-01
