Observed this also for noble:linux 6.8.0-51.52 on s390x metal (kernel06) in security cycle s2024.10.28.
-- You received this bug notification because you are a member of Canonical Platform QA Team, which is subscribed to ubuntu-kernel-tests. https://bugs.launchpad.net/bugs/2072501 Title: net:pmtu.sh of ubuntu_kselftests_net and subsequent subtests time out Status in ubuntu-kernel-tests: New Bug description: Found on N-nvidia-64k with node hinyari. After net:pmtu.sh times out, these tests also timeout until "TEST SYSTEM FAILURE DETECTED" is eventually triggered: net:udpgso.sh, net:ip_defrag.sh, net:udpgso_bench.sh, net:fib_rule_tests.sh. 14:44:14 INFO | START ubuntu_kselftests_net.net:pmtu.sh ubuntu_kselftests_net.net:pmtu.sh timeout=2700 timestamp=1719413054 localtime=Jun 26 14:44:14 14:44:14 DEBUG| Persistent state client._record_indent now set to 2 14:44:14 DEBUG| Persistent state client.unexpected_reboot now set to ('ubuntu_kselftests_net.net:pmtu.sh', 'ubuntu_kselftests_net.net:pmtu.sh') 14:44:14 DEBUG| Waiting for pid 43936 for 2700 seconds 14:44:14 WARNI| System python is too old, crash handling disabled 14:44:14 DEBUG| Running 'make run_tests -C net TEST_PROGS=pmtu.sh TEST_GEN_PROGS='' TEST_CUSTOM_PROGS=''' 14:44:14 DEBUG| [stdout] make: Entering directory '/home/ubuntu/autotest/client/tmp/ubuntu_kselftests_net/src/linux/tools/testing/selftests/net' 14:44:14 DEBUG| [stdout] TAP version 13 14:44:14 DEBUG| [stdout] 1..1 14:44:14 DEBUG| [stdout] # timeout set to 0 14:44:14 DEBUG| [stdout] # selftests: net: pmtu.sh 14:44:14 DEBUG| [stdout] # TEST: ipv4: PMTU exceptions [ OK ] 14:44:14 DEBUG| [stdout] # TEST: ipv4: PMTU exceptions - nexthop objects [ OK ] 14:44:14 DEBUG| [stdout] # TEST: ipv6: PMTU exceptions [ OK ] 14:44:14 DEBUG| [stdout] # TEST: ipv6: PMTU exceptions - nexthop objects [ OK ] 14:44:14 DEBUG| [stdout] # TEST: ICMPv4 with DSCP and ECN: PMTU exceptions [ OK ] 14:44:15 DEBUG| [stdout] # TEST: ICMPv4 with DSCP and ECN: PMTU exceptions - nexthop objects [ OK ] 14:44:15 DEBUG| [stdout] # 2024/06/26 14:44:15 socat[44926] E xioopen_ipdgram_listen(): unknown address family 0 14:44:15 DEBUG| [stdout] # TEST: UDPv4 with DSCP and ECN: PMTU exceptions [ OK ] 14:44:15 DEBUG| [stdout] # ./pmtu.sh: line 937: kill: (44926) - No such process 14:44:15 DEBUG| [stdout] # 2024/06/26 14:44:15 socat[45066] E xioopen_ipdgram_listen(): unknown address family 0 14:44:15 DEBUG| [stdout] # TEST: UDPv4 with DSCP and ECN: PMTU exceptions - nexthop objects [ OK ] 14:44:15 DEBUG| [stdout] # ./pmtu.sh: line 937: kill: (45066) - No such process 14:44:16 DEBUG| [stdout] # TEST: IPv4 over vxlan4: PMTU exceptions [ OK ] 14:44:17 DEBUG| [stdout] # TEST: IPv4 over vxlan4: PMTU exceptions - nexthop objects [ OK ] 14:44:18 DEBUG| [stdout] # TEST: IPv6 over vxlan4: PMTU exceptions [ OK ] 14:44:19 DEBUG| [stdout] # TEST: IPv6 over vxlan4: PMTU exceptions - nexthop objects [ OK ] 14:44:20 DEBUG| [stdout] # TEST: IPv4 over vxlan6: PMTU exceptions [ OK ] 14:44:22 DEBUG| [stdout] # TEST: IPv4 over vxlan6: PMTU exceptions - nexthop objects [ OK ] 14:44:23 DEBUG| [stdout] # TEST: IPv6 over vxlan6: PMTU exceptions [ OK ] 14:44:24 DEBUG| [stdout] # TEST: IPv6 over vxlan6: PMTU exceptions - nexthop objects [ OK ] 14:44:25 DEBUG| [stdout] # TEST: IPv4 over geneve4: PMTU exceptions [ OK ] 14:44:26 DEBUG| [stdout] # TEST: IPv4 over geneve4: PMTU exceptions - nexthop objects [ OK ] 14:44:27 DEBUG| [stdout] # TEST: IPv6 over geneve4: PMTU exceptions [ OK ] 14:44:28 DEBUG| [stdout] # TEST: IPv6 over geneve4: PMTU exceptions - nexthop objects [ OK ] 14:44:30 DEBUG| [stdout] # TEST: IPv4 over geneve6: PMTU exceptions [ OK ] 14:44:31 DEBUG| [stdout] # TEST: IPv4 over geneve6: PMTU exceptions - nexthop objects [ OK ] 14:44:32 DEBUG| [stdout] # TEST: IPv6 over geneve6: PMTU exceptions [ OK ] 14:44:33 DEBUG| [stdout] # TEST: IPv6 over geneve6: PMTU exceptions - nexthop objects [ OK ] 14:44:35 DEBUG| [stdout] # TEST: IPv4, bridged vxlan4: PMTU exceptions [ OK ] 14:44:37 DEBUG| [stdout] # TEST: IPv4, bridged vxlan4: PMTU exceptions - nexthop objects [ OK ] 14:44:40 DEBUG| [stdout] # TEST: IPv6, bridged vxlan4: PMTU exceptions [ OK ] 14:44:42 DEBUG| [stdout] # TEST: IPv6, bridged vxlan4: PMTU exceptions - nexthop objects [ OK ] 14:44:44 DEBUG| [stdout] # TEST: IPv4, bridged vxlan6: PMTU exceptions [ OK ] 14:44:46 DEBUG| [stdout] # TEST: IPv4, bridged vxlan6: PMTU exceptions - nexthop objects [ OK ] 14:44:49 DEBUG| [stdout] # TEST: IPv6, bridged vxlan6: PMTU exceptions [ OK ] 14:44:51 DEBUG| [stdout] # TEST: IPv6, bridged vxlan6: PMTU exceptions - nexthop objects [ OK ] 14:44:53 DEBUG| [stdout] # TEST: IPv4, bridged geneve4: PMTU exceptions [ OK ] 14:44:55 DEBUG| [stdout] # TEST: IPv4, bridged geneve4: PMTU exceptions - nexthop objects [ OK ] 14:44:58 DEBUG| [stdout] # TEST: IPv6, bridged geneve4: PMTU exceptions [ OK ] 14:45:00 DEBUG| [stdout] # TEST: IPv6, bridged geneve4: PMTU exceptions - nexthop objects [ OK ] 14:45:02 DEBUG| [stdout] # TEST: IPv4, bridged geneve6: PMTU exceptions [ OK ] 14:45:04 DEBUG| [stdout] # TEST: IPv4, bridged geneve6: PMTU exceptions - nexthop objects [ OK ] 14:45:07 DEBUG| [stdout] # TEST: IPv6, bridged geneve6: PMTU exceptions [ OK ] 14:45:09 DEBUG| [stdout] # TEST: IPv6, bridged geneve6: PMTU exceptions - nexthop objects [ OK ] 14:45:09 DEBUG| [stdout] # ovs_bridge not supported 14:45:09 DEBUG| [stdout] # TEST: IPv4, OVS vxlan4: PMTU exceptions [SKIP] 14:45:09 DEBUG| [stdout] # ovs_bridge not supported 14:45:09 DEBUG| [stdout] # TEST: IPv6, OVS vxlan4: PMTU exceptions [SKIP] 14:45:09 DEBUG| [stdout] # ovs_bridge not supported 14:45:09 DEBUG| [stdout] # TEST: IPv4, OVS vxlan6: PMTU exceptions [SKIP] 14:45:09 DEBUG| [stdout] # ovs_bridge not supported 14:45:09 DEBUG| [stdout] # TEST: IPv6, OVS vxlan6: PMTU exceptions [SKIP] 14:45:09 DEBUG| [stdout] # ovs_bridge not supported 14:45:09 DEBUG| [stdout] # TEST: IPv4, OVS geneve4: PMTU exceptions [SKIP] 14:45:10 DEBUG| [stdout] # ovs_bridge not supported 14:45:10 DEBUG| [stdout] # TEST: IPv6, OVS geneve4: PMTU exceptions [SKIP] 14:45:10 DEBUG| [stdout] # ovs_bridge not supported 14:45:10 DEBUG| [stdout] # TEST: IPv4, OVS geneve6: PMTU exceptions [SKIP] 14:45:10 DEBUG| [stdout] # ovs_bridge not supported 14:45:10 DEBUG| [stdout] # TEST: IPv6, OVS geneve6: PMTU exceptions [SKIP] 14:45:11 DEBUG| [stdout] # TEST: IPv4 over fou4: PMTU exceptions [ OK ] 14:45:11 DEBUG| [stdout] # TEST: IPv4 over fou4: PMTU exceptions - nexthop objects [ OK ] 14:45:11 DEBUG| [stdout] # TEST: IPv6 over fou4: PMTU exceptions [ OK ] 14:45:12 DEBUG| [stdout] # TEST: IPv6 over fou4: PMTU exceptions - nexthop objects [ OK ] 14:45:12 DEBUG| [stdout] # TEST: IPv4 over fou6: PMTU exceptions [ OK ] 14:45:12 DEBUG| [stdout] # TEST: IPv4 over fou6: PMTU exceptions - nexthop objects [ OK ] 14:45:12 DEBUG| [stdout] # TEST: IPv6 over fou6: PMTU exceptions [ OK ] 14:45:13 DEBUG| [stdout] # TEST: IPv6 over fou6: PMTU exceptions - nexthop objects [ OK ] 14:45:13 DEBUG| [stdout] # TEST: IPv4 over gue4: PMTU exceptions [ OK ] 14:45:13 DEBUG| [stdout] # TEST: IPv4 over gue4: PMTU exceptions - nexthop objects [ OK ] 14:45:13 DEBUG| [stdout] # TEST: IPv6 over gue4: PMTU exceptions [ OK ] 14:45:14 DEBUG| [stdout] # TEST: IPv6 over gue4: PMTU exceptions - nexthop objects [ OK ] 14:45:14 DEBUG| [stdout] # TEST: IPv4 over gue6: PMTU exceptions [ OK ] 14:45:14 DEBUG| [stdout] # TEST: IPv4 over gue6: PMTU exceptions - nexthop objects [ OK ] 14:45:14 DEBUG| [stdout] # TEST: IPv6 over gue6: PMTU exceptions [ OK ] 14:45:14 DEBUG| [stdout] # TEST: IPv6 over gue6: PMTU exceptions - nexthop objects [ OK ] 14:45:15 DEBUG| [stdout] # TEST: IPv4 over IPv4: PMTU exceptions [ OK ] 14:45:15 DEBUG| [stdout] # TEST: IPv4 over IPv4: PMTU exceptions - nexthop objects [ OK ] 14:45:15 DEBUG| [stdout] # TEST: IPv6 over IPv4: PMTU exceptions [ OK ] 14:45:15 DEBUG| [stdout] # TEST: IPv6 over IPv4: PMTU exceptions - nexthop objects [ OK ] 14:45:15 DEBUG| [stdout] # TEST: IPv4 over IPv6: PMTU exceptions [ OK ] 14:45:16 DEBUG| [stdout] # TEST: IPv4 over IPv6: PMTU exceptions - nexthop objects [ OK ] 14:45:16 DEBUG| [stdout] # TEST: IPv6 over IPv6: PMTU exceptions [ OK ] 14:45:16 DEBUG| [stdout] # TEST: IPv6 over IPv6: PMTU exceptions - nexthop objects [ OK ] 15:29:14 INFO | Timer expired (2700 sec.), nuking pid 43936 15:29:14 INFO | ERROR ubuntu_kselftests_net.net:pmtu.sh ubuntu_kselftests_net.net:pmtu.sh timestamp=1719415754 localtime=Jun 26 15:29:14 Test timeout expired, rc=15 15:29:14 INFO | END ERROR ubuntu_kselftests_net.net:pmtu.sh ubuntu_kselftests_net.net:pmtu.sh timestamp=1719415754 localtime=Jun 26 15:29:14 15:29:14 DEBUG| Persistent state client._record_indent now set to 1 15:29:14 DEBUG| Persistent state client.unexpected_reboot deleted 15:29:14 DEBUG| Test has timeout: 2700 sec. 15:29:14 INFO | START ubuntu_kselftests_net.net:udpgso.sh ubuntu_kselftests_net.net:udpgso.sh timeout=2700 timestamp=1719415754 localtime=Jun 26 15:29:14 15:29:14 DEBUG| Persistent state client._record_indent now set to 2 15:29:14 DEBUG| Persistent state client.unexpected_reboot now set to ('ubuntu_kselftests_net.net:udpgso.sh', 'ubuntu_kselftests_net.net:udpgso.sh') 15:29:14 DEBUG| Waiting for pid 54956 for 2700 seconds 15:29:14 WARNI| System python is too old, crash handling disabled 15:29:14 DEBUG| Running 'make run_tests -C net TEST_PROGS=udpgso.sh TEST_GEN_PROGS='' TEST_CUSTOM_PROGS=''' 15:29:14 DEBUG| [stdout] make: Entering directory '/home/ubuntu/autotest/client/tmp/ubuntu_kselftests_net/src/linux/tools/testing/selftests/net' 15:29:14 DEBUG| [stdout] TAP version 13 15:29:14 DEBUG| [stdout] 1..1 15:29:14 DEBUG| [stdout] # timeout set to 0 15:29:14 DEBUG| [stdout] # selftests: net: udpgso.sh 15:29:14 DEBUG| [stdout] # ipv4 cmsg 16:14:14 INFO | Timer expired (2700 sec.), nuking pid 54956 16:14:14 INFO | ERROR ubuntu_kselftests_net.net:udpgso.sh ubuntu_kselftests_net.net:udpgso.sh timestamp=1719418454 localtime=Jun 26 16:14:14 Test timeout expired, rc=15 16:14:14 INFO | END ERROR ubuntu_kselftests_net.net:udpgso.sh ubuntu_kselftests_net.net:udpgso.sh timestamp=1719418454 localtime=Jun 26 ... Accompanied by the following dmesg entries, "waiting for veth_A-R1 to become free" appears to repeat until the system is shutdown: [ 574.404587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 584.652587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 594.900587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 605.148587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 615.396587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 625.644587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 635.892587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 646.140587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 656.388587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 666.636587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 676.884586] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 687.132586] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 697.380587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 707.628586] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 717.876586] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 728.124587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 738.284631] INFO: task modprobe:54505 blocked for more than 122 seconds. [ 738.291488] Not tainted 6.8.0-1009-nvidia-64k #9-Ubuntu [ 738.297364] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 738.372587] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 [ 748.620586] unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 5 ... To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2072501/+subscriptions -- Mailing list: https://launchpad.net/~canonical-ubuntu-qa Post to : canonical-ubuntu-qa@lists.launchpad.net Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa More help : https://help.launchpad.net/ListHelp
