Folks,
Due to the upgrade to ubuntu-22.04 and re-enabling multi-worker tests on
debian-11 CI jobs, there have been a number of regressions in 'make
test' (some intermittent failures, other hard failures).
I have disabled the offending tests [0] and rebased all of the
outstanding gerrit changes which had failed due to these 'make test'
failures. See below list of jobs skipped on debian-11 [1] and
ubuntu-22.04 [2]
I will be reaching out to maintainers to help troubleshoot, fix, and
re-enable all of these tests and will plan to discuss this effort at
tomorrow's VPP Community Meeting
Thanks,
-daw-
[0] https://gerrit.fd.io/r/c/vpp/+/37198
https://gerrit.fd.io/r/c/vpp/+/37144
[1] Tests Disabled on Debian-11:
----- %< -----
fixme-on-debian11-console-timestamp.log:01:40:53 FIXME on Debian-11:
Wireguard Test Case [2 worker threads]
fixme-on-debian11-console-timestamp.log:01:40:53 FIXME on Debian-11:
Wireguard Tests in multi worker setup [2 worker threads]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - rekey [main thread only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - rekey repeat [main thread
only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - pre shared key auth [main
thread only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - rekey with key exchange
[main thread only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: Dead peer detection test [main thread only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - rekey repeat with key
exchange [main thread only]
fixme-on-debian11-console-timestamp.log:02:04:28 FIXME on Debian-11:
FIXME with VPP workers: test ikev2 responder - non-default table id
[main thread only]
fixme-on-debian11-console-timestamp.log:02:08:18 FIXME on Debian-11:
NAT44EI Test Cases [2 worker threads]
fixme-on-debian11-console-timestamp.log:02:08:44 FIXME on Debian-11:
SOLO RUN: Memif Test Case [2 worker threads]
fixme-on-debian11-console-timestamp.log:02:09:54 FIXME on Debian-11:
SOLO RUN: Bidirectional Forwarding Detection (BFD) [2 worker threads]
fixme-on-debian11-console-timestamp.log:02:13:19 FIXME on Debian-11:
FIXME with VPP workers: SOLO RUN: Template verification, timer tests
[main thread only]
----- %< -----
[2] Tests Disabled on Ubuntu-22.04:
----- %< -----
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARon 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARon 3DES-CBC/SHA1-96 IPSec test [main thread
only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARon 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNon ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:53 FIXME on
Ubuntu-22.04: native ESNoff ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:55 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:55 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:38:59 FIXME on
Ubuntu-22.04: ipsecmb ESNon ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:11 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARon 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:15 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:15 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:15 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:23 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:23 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:24 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:24 FIXME on
Ubuntu-22.04: ipsecmb ESNoff ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:41 FIXME on
Ubuntu-22.04: openssl ESNon ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:44 FIXME on
Ubuntu-22.04: openssl ESNon ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:44 FIXME on
Ubuntu-22.04: openssl ESNon ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:39:44 FIXME on
Ubuntu-22.04: openssl ESNon ARon 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:09 FIXME on
Ubuntu-22.04: openssl ESNon ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:09 FIXME on
Ubuntu-22.04: openssl ESNon ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:10 FIXME on
Ubuntu-22.04: openssl ESNon ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:10 FIXME on
Ubuntu-22.04: openssl ESNon ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:26 FIXME on
Ubuntu-22.04: openssl ESNoff ARon 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:26 FIXME on
Ubuntu-22.04: openssl ESNoff ARon AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:35 FIXME on
Ubuntu-22.04: openssl ESNoff ARon AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:35 FIXME on
Ubuntu-22.04: openssl ESNoff ARon AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:49 FIXME on
Ubuntu-22.04: openssl ESNoff ARoff 3DES-CBC/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:49 FIXME on
Ubuntu-22.04: openssl ESNoff ARoff AES-CBC-192/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:49 FIXME on
Ubuntu-22.04: openssl ESNoff ARoff AES-CBC-256/SHA1-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:40:49 FIXME on
Ubuntu-22.04: openssl ESNoff ARoff AES-CBC-128/MD5-96 IPSec test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:41:14 FIXME on
Ubuntu-22.04: async ESNon ARon AES-CBC-192/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:15 FIXME on
Ubuntu-22.04: async ESNon ARon AES-CBC-256/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:26 FIXME on
Ubuntu-22.04: async ESNon ARoff AES-CBC-192/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:26 FIXME on
Ubuntu-22.04: async ESNon ARoff AES-CBC-256/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:40 FIXME on
Ubuntu-22.04: async ESNoff ARoff AES-CBC-256/SHA1-96 IPSec test [2
worker threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:40 FIXME on
Ubuntu-22.04: async ESNoff ARon AES-CBC-256/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:40 FIXME on
Ubuntu-22.04: async ESNoff ARon AES-CBC-192/SHA1-96 IPSec test [2 worker
threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:40 FIXME on
Ubuntu-22.04: async ESNoff ARoff AES-CBC-192/SHA1-96 IPSec test [2
worker threads]
fixme-on-ubuntu2204-console-timestamp.log:01:41:45 FIXME on
Ubuntu-22.04: Wireguard Test Case [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:41:47 FIXME on
Ubuntu-22.04: Wireguard Tests in multi worker setup [2 worker threads]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - pre shared
key auth [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: Dead peer detection test [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - non-default
table id [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - rekey
repeat [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - rekey
repeat with key exchange [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:36 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - rekey with
key exchange [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:44:40 FIXME on
Ubuntu-22.04: FIXME with VPP workers: test ikev2 responder - rekey [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Early Qclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Early
Sclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo Internal Transfer Test
Case [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Sclose
Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Big
Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Early
Qclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo Internal Serial Transfer
Test Case [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Sclose
Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Qclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Early Qclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Big Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Early
Qclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Sclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:20 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo Internal MultiStream
Test Case [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Early
Sclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Qclose
Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Qclose
Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo Ext Transfer Server
Stream Early Sclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Early Sclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Test
Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Sclose Tx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream MultiWorker Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:46:24 FIXME on
Ubuntu-22.04: FIXME with VPP workers: QUIC Echo External Transfer Server
Stream Qclose Rx Test Case [1 worker thread]
fixme-on-ubuntu2204-console-timestamp.log:01:47:11 FIXME on
Ubuntu-22.04: ARP/ND Aging [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:47:28 FIXME on
Ubuntu-22.04: FIXME with VPP workers: NAT64 Test Cases [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:48:16 FIXME on
Ubuntu-22.04: IPsec IPv4 Tunnel interface all Algos [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:49:54 FIXME on
Ubuntu-22.04: NAT44ED MW Test Case [4 worker threads]
fixme-on-ubuntu2204-console-timestamp.log:01:53:31 FIXME on
Ubuntu-22.04: SOLO RUN: Bidirectional Forwarding Detection (BFD) [main
thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:53:32 FIXME on
Ubuntu-22.04: FIXME with VPP workers: SOLO RUN: Bidirectional Forwarding
Detection (BFD) (IPv6) [main thread only]
fixme-on-ubuntu2204-console-timestamp.log:01:56:00 FIXME on
Ubuntu-22.04: FIXME with VPP workers: SOLO RUN: Template verification,
timer tests [main thread only]
----- %< -----
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21932): https://lists.fd.io/g/vpp-dev/message/21932
Mute This Topic: https://lists.fd.io/mt/93943475/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
