Eric, could this possibly be commit 4179b00c04d1 ("geneve: pull IP header before ECN decapsulation")?
On Wed, 9 Dec 2020 10:05:14 +0800 Jianlin Shi wrote: > Hi , > > I reported a bug in bugzilla.kernel.org for geneve issue: > https://bugzilla.kernel.org/show_bug.cgi?id=210569 > > Thanks & Best Regards, > Jianlin Shi > > > On Tue, Dec 8, 2020 at 9:38 AM Jianlin Shi <ji...@redhat.com> wrote: > > > > Hi , > > > > > > On Tue, Dec 8, 2020 at 8:25 AM CKI Project <cki-proj...@redhat.com> wrote: > >> > >> > >> Hello, > >> > >> We ran automated tests on a recent commit from this kernel tree: > >> > >> Kernel repo: > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > >> Commit: 7059c2c00a21 - Merge branch 'for-linus' of > >> git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input > >> > >> The results of these automated tests are provided below. > >> > >> Overall result: FAILED (see details below) > >> Merge: OK > >> Compile: OK > >> Selftests compile: FAILED > >> Tests: FAILED > >> > >> Pipeline: > >> https://xci32.lab.eng.rdu2.redhat.com/cki-project/cki-pipeline/-/pipelines/619303 > >> > >> Check out our dashboard including known failures tagged by our bot at > >> https://datawarehouse.internal.cki-project.org/kcidb/revisions/7748 > >> > >> One or more kernel tests failed: > >> > >> s390x: > >> ❌ LTP > >> ❌ Networking tunnel: geneve basic test > >> > >> ppc64le: > >> ❌ Boot test > >> ❌ LTP > >> ❌ Networking tunnel: geneve basic test > >> > >> aarch64: > >> ❌ storage: software RAID testing > >> ❌ LTP > >> ❌ Networking tunnel: geneve basic test > >> > >> x86_64: > >> ❌ LTP > >> ❌ Networking tunnel: geneve basic test > > > > > > the basic traffic over geneve would fail, could you help to check. Should > > we report a bug or not? > > > > > > ip netns add client > > > > ip netns add server > > > > > > > > ip link add veth0_c type veth peer name veth0_s > > > > ip link set veth0_c netns client > > > > ip link set veth0_s netns server > > > > > > > > ip netns exec client ip link set lo up > > > > ip netns exec client ip link set veth0_c up > > > > > > > > ip netns exec server ip link set lo up > > > > ip netns exec server ip link set veth0_s up > > > > > > > > > > > > ip netns exec client ip addr add 2000::1/64 dev veth0_c > > > > ip netns exec client ip addr add 10.10.0.1/24 dev veth0_c > > > > > > > > ip netns exec server ip addr add 2000::2/64 dev veth0_s > > > > ip netns exec server ip addr add 10.10.0.2/24 dev veth0_s > > > > > > > > ip netns exec client ping 10.10.0.2 -c 2 > > > > ip netns exec client ping6 2000::2 -c 2 > > > > > > > > ip netns exec client ip link add geneve1 type geneve vni 1234 remote > > 10.10.0.2 ttl 64 > > > > ip netns exec server ip link add geneve1 type geneve vni 1234 remote > > 10.10.0.1 ttl 64 > > > > > > > > ip netns exec client ip link set geneve1 up > > > > ip netns exec client ip addr add 1.1.1.1/24 dev geneve1 > > > > ip netns exec server ip link set geneve1 up > > > > ip netns exec server ip addr add 1.1.1.2/24 dev geneve1 > > ip netns exec client ping 1.1.1.2 -c 3 > > > > > >> > >> ❌ storage: software RAID testing > >> > >> We hope that these logs can help you find the problem quickly. For the full > >> detail on our testing procedures, please scroll to the bottom of this > >> message. > >> > >> Please reply to this email if you have any questions about the tests that > >> we > >> ran or if you have any suggestions on how to make future tests more > >> effective. > >> > >> ,-. ,-. > >> ( C ) ( K ) Continuous > >> `-',-.`-' Kernel > >> ( I ) Integration > >> `-' > >> ______________________________________________________________________________ > >> > >> Compile testing > >> --------------- > >> > >> We compiled the kernel for 4 architectures: > >> > >> aarch64: > >> make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > >> > >> ppc64le: > >> make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > >> > >> s390x: > >> make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > >> > >> x86_64: > >> make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > >> > >> > >> We built the following selftests: > >> > >> x86_64: > >> net: OK > >> bpf: fail > >> install and packaging: OK > >> > >> You can find the full log (build-selftests.log) in the artifact storage > >> above. > >> > >> > >> Hardware testing > >> ---------------- > >> All the testing jobs are listed here: > >> > >> > >> https://beaker.engineering.redhat.com/jobs/?jobsearch-0.table=Whiteboard&jobsearch-0.operation=contains&jobsearch-0.value=cki%40gitlab%3A619303 > >> > >> We booted each kernel and ran the following tests: > >> > >> aarch64: > >> Host 1: https://beaker.engineering.redhat.com/recipes/9156933 > >> ✅ Boot test > >> ✅ selinux-policy: serge-testsuite > >> ❌ storage: software RAID testing > >> ✅ stress: stress-ng > >> ❌ xfstests - ext4 > >> ✅ xfstests - xfs > >> ✅ xfstests - btrfs > >> ❌ IPMI driver test > >> ✅ IPMItool loop stress test > >> ✅ Storage blktests > >> ✅ Storage block - filesystem fio test > >> ✅ Storage block - queue scheduler test > >> ✅ Storage nvme - tcp > >> ✅ Storage: swraid mdadm raid_module test > >> > >> Host 2: https://beaker.engineering.redhat.com/recipes/9156932 > >> ✅ Boot test > >> ✅ ACPI table test > >> ✅ ACPI enabled test > >> ❌ LTP > >> ✅ Loopdev Sanity > >> ✅ Memory: fork_mem > >> ✅ Memory function: memfd_create > >> ✅ AMTU (Abstract Machine Test Utility) > >> ✅ Networking bridge: sanity > >> ✅ Networking socket: fuzz > >> ✅ Networking: igmp conformance test > >> ✅ Networking route: pmtu > >> ✅ Networking route_func - local > >> ✅ Networking route_func - forward > >> ✅ Networking TCP: keepalive test > >> ✅ Networking UDP: socket > >> ❌ Networking tunnel: geneve basic test > >> ✅ Networking tunnel: gre basic > >> ✅ L2TP basic test > >> ✅ Networking tunnel: vxlan basic > >> ✅ Networking ipsec: basic netns - transport > >> ✅ Networking ipsec: basic netns - tunnel > >> ✅ Libkcapi AF_ALG test > >> ✅ pciutils: update pci ids test > >> ✅ ALSA PCM loopback test > >> ✅ ALSA Control (mixer) Userspace Element test > >> ✅ storage: SCSI VPD > >> ✅ CIFS Connectathon > >> ✅ POSIX pjd-fstest suites > >> ✅ Firmware test suite > >> ✅ jvm - jcstress tests > >> ✅ Memory function: kaslr > >> ✅ Ethernet drivers sanity > >> ✅ Networking firewall: basic netfilter test > >> ✅ audit: audit testsuite test > >> ✅ trace: ftrace/tracer > >> ✅ kdump - kexec_boot > >> > >> ppc64le: > >> Host 1: https://beaker.engineering.redhat.com/recipes/9156935 > >> ❌ Boot test > >> ⚡⚡⚡ selinux-policy: serge-testsuite > >> ⚡⚡⚡ storage: software RAID testing > >> ⚡⚡⚡ xfstests - ext4 > >> ⚡⚡⚡ xfstests - xfs > >> ⚡⚡⚡ xfstests - btrfs > >> ⚡⚡⚡ IPMI driver test > >> ⚡⚡⚡ IPMItool loop stress test > >> ⚡⚡⚡ Storage blktests > >> ⚡⚡⚡ Storage block - filesystem fio test > >> ⚡⚡⚡ Storage block - queue scheduler test > >> ⚡⚡⚡ Storage nvme - tcp > >> ⚡⚡⚡ Storage: swraid mdadm raid_module test > >> > >> Host 2: https://beaker.engineering.redhat.com/recipes/9156934 > >> ✅ Boot test > >> ❌ LTP > >> ✅ Loopdev Sanity > >> ✅ Memory: fork_mem > >> ✅ Memory function: memfd_create > >> ✅ AMTU (Abstract Machine Test Utility) > >> ✅ Networking bridge: sanity > >> ✅ Networking socket: fuzz > >> ✅ Networking route: pmtu > >> ✅ Networking route_func - local > >> ✅ Networking route_func - forward > >> ✅ Networking TCP: keepalive test > >> ✅ Networking UDP: socket > >> ❌ Networking tunnel: geneve basic test > >> ✅ Networking tunnel: gre basic > >> ✅ L2TP basic test > >> ✅ Networking tunnel: vxlan basic > >> ✅ Networking ipsec: basic netns - tunnel > >> ✅ Libkcapi AF_ALG test > >> ✅ pciutils: update pci ids test > >> ✅ ALSA PCM loopback test > >> ✅ ALSA Control (mixer) Userspace Element test > >> ✅ CIFS Connectathon > >> ✅ POSIX pjd-fstest suites > >> ✅ jvm - jcstress tests > >> ✅ Memory function: kaslr > >> ✅ Ethernet drivers sanity > >> ✅ Networking firewall: basic netfilter test > >> ✅ audit: audit testsuite test > >> ✅ trace: ftrace/tracer > >> > >> s390x: > >> Host 1: https://beaker.engineering.redhat.com/recipes/9156940 > >> ✅ Boot test > >> ✅ selinux-policy: serge-testsuite > >> ✅ stress: stress-ng > >> ✅ Storage blktests > >> ❌ Storage nvme - tcp > >> ✅ Storage: swraid mdadm raid_module test > >> > >> Host 2: https://beaker.engineering.redhat.com/recipes/9156939 > >> ✅ Boot test > >> ❌ LTP > >> ✅ Loopdev Sanity > >> ✅ Memory: fork_mem > >> ✅ Memory function: memfd_create > >> ✅ AMTU (Abstract Machine Test Utility) > >> ✅ Networking bridge: sanity > >> ✅ Networking route: pmtu > >> ✅ Networking route_func - local > >> ✅ Networking route_func - forward > >> ✅ Networking TCP: keepalive test > >> ✅ Networking UDP: socket > >> ❌ Networking tunnel: geneve basic test > >> ✅ Networking tunnel: gre basic > >> ✅ L2TP basic test > >> ✅ Networking tunnel: vxlan basic > >> ✅ Networking ipsec: basic netns - transport > >> ✅ Networking ipsec: basic netns - tunnel > >> ✅ Libkcapi AF_ALG test > >> ✅ CIFS Connectathon > >> ✅ POSIX pjd-fstest suites > >> ✅ jvm - jcstress tests > >> ✅ Memory function: kaslr > >> ✅ Ethernet drivers sanity > >> ✅ Networking firewall: basic netfilter test > >> ❌ audit: audit testsuite test > >> ✅ trace: ftrace/tracer > >> > >> x86_64: > >> Host 1: https://beaker.engineering.redhat.com/recipes/9156936 > >> ✅ Boot test > >> ✅ ACPI table test > >> ❌ LTP > >> ✅ Loopdev Sanity > >> ✅ Memory: fork_mem > >> ✅ Memory function: memfd_create > >> ✅ AMTU (Abstract Machine Test Utility) > >> ✅ Networking bridge: sanity > >> ✅ Networking socket: fuzz > >> ✅ Networking: igmp conformance test > >> ✅ Networking route: pmtu > >> ✅ Networking route_func - local > >> ✅ Networking route_func - forward > >> ✅ Networking TCP: keepalive test > >> ✅ Networking UDP: socket > >> ❌ Networking tunnel: geneve basic test > >> ✅ Networking tunnel: gre basic > >> ✅ L2TP basic test > >> ✅ Networking tunnel: vxlan basic > >> ✅ Networking ipsec: basic netns - transport > >> ✅ Networking ipsec: basic netns - tunnel > >> ✅ Libkcapi AF_ALG test > >> ✅ pciutils: sanity smoke test > >> ✅ pciutils: update pci ids test > >> ✅ ALSA PCM loopback test > >> ✅ ALSA Control (mixer) Userspace Element test > >> ✅ storage: SCSI VPD > >> ✅ CIFS Connectathon > >> ✅ POSIX pjd-fstest suites > >> ✅ Firmware test suite > >> ✅ jvm - jcstress tests > >> ✅ Memory function: kaslr > >> ✅ Ethernet drivers sanity > >> ✅ Networking firewall: basic netfilter test > >> ✅ audit: audit testsuite test > >> ✅ trace: ftrace/tracer > >> ✅ kdump - kexec_boot > >> > >> Host 2: https://beaker.engineering.redhat.com/recipes/9156937 > >> ✅ Boot test > >> ✅ kdump - sysrq-c > >> ✅ kdump - file-load > >> > >> Host 3: https://beaker.engineering.redhat.com/recipes/9156938 > >> > >> ⚡ Internal infrastructure issues prevented one or more tests (marked > >> with ⚡⚡⚡) from running on this architecture. > >> This is not the fault of the kernel that was tested. > >> > >> ✅ Boot test > >> ✅ selinux-policy: serge-testsuite > >> ❌ storage: software RAID testing > >> ✅ stress: stress-ng > >> ❌ CPU: Frequency Driver Test > >> ✅ CPU: Idle Test > >> ❌ xfstests - ext4 > >> ✅ xfstests - xfs > >> ✅ xfstests - btrfs > >> ⚡⚡⚡ IPMI driver test > >> ⚡⚡⚡ IPMItool loop stress test > >> ⚡⚡⚡ power-management: cpupower/sanity test > >> ⚡⚡⚡ Storage blktests > >> ⚡⚡⚡ Storage block - filesystem fio test > >> ⚡⚡⚡ Storage block - queue scheduler test > >> ⚡⚡⚡ Storage nvme - tcp > >> ⚡⚡⚡ Storage: swraid mdadm raid_module test > >> > >> Test sources: https://gitlab.com/cki-project/kernel-tests > >> Pull requests are welcome for new tests or improvements to existing > >> tests! > >> > >> Aborted tests > >> ------------- > >> Tests that didn't complete running successfully are marked with ⚡⚡⚡. > >> If this was caused by an infrastructure issue, we try to mark that > >> explicitly in the report. > >> > >> Waived tests > >> ------------ > >> If the test run included waived tests, they are marked with . Such tests > >> are > >> executed but their results are not taken into account. Tests are waived > >> when > >> their results are not reliable enough, e.g. when they're just introduced > >> or are > >> being fixed. > >> > >> Testing timeout > >> --------------- > >> We aim to provide a report within reasonable timeframe. Tests that haven't > >> finished running yet are marked with ⏱. > >> > >> Reproducing results > >> ------------------- > >> Click on a link below to access a web page that allows you to adjust the > >> Beaker job and re-run any failed tests. These links are generated for > >> failed or aborted tests that are not waived. Please adjust the Beaker > >> job whiteboard string in the web page so that it is easy for you to find > >> and so that it is not confused with a regular CKI job. > >> > >> After clicking the "Submit the job!" button, a dialog will open that should > >> contain a link to the newly submitted Beaker job. > >> > >> > >> https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794082&recipe_id=9156933&recipe_id=9156932&job_id=4794082 > >> > >> https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794083&recipe_id=9156935&recipe_id=9156934&job_id=4794083 > >> > >> https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794085&recipe_id=9156939&job_id=4794085 > >> > >> https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794084&recipe_id=9156936&recipe_id=9156938&job_id=4794084 > >> >