Hey Jon from Nutanix AHV R&D here - We also see this issue on RHEL, SLE15, and vanilla kernels. In all three of those cases, the problem wasn't with AHV, but with a problematic set of backport that was tagged as a CVE, but caused a GSO-based performance regression. This can be reproduced on non-Nutanix AHV hosts too.
Cross-referencing: RHEL bugs: https://issues.redhat.com/browse/RHEL-62411 https://issues.redhat.com/browse/RHEL-67427 CentOS stream PR to fix this: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/5623 Generic reproduction from the Tailscale community, lot of great info in this thread: https://github.com/tailscale/tailscale/issues/13041#issuecomment-2294011607 Cross reference for SUSE SLES15 SP6: - broken in SLE15 SP6 with kernel 6.4.0-150600.23.22.1 (Sep-23-2024) - fixed in SLE15 SP6 with kernel 6.4.0-150600.23.25.1 (Oct-09-2024) - Versions are tracked here https://www.suse.com/support/kb/doc/?id=000019587 and aligned to the commit tags here: - bad commit: https://github.com/SUSE/kernel/commit/25229b4457540a2da59598839f39f8fbe3dfaa64 - good commit: https://github.com/SUSE/kernel/commit/e5010362f58efcc32c223cae45c7939ed3d04399 TLDR, the breaking commit is: net: drop bad gso csum_start and offset in virtio_net_hdr See commits with Fixes tag: Fixes: 89add40066f9 ("net: drop bad gso csum_start and offset in virtio_net_hdr") Fixing commits in our other cases with RHEL, SLE15, and generic vanilla upstream kernels: udp: fix receiving fraglist GSO packets net: tighten bad gso csum offset check in virtio_net_hdr Any chance this specific ubuntu kernel has "net: drop bad gso csum_start and offset in virtio_net_hdr" and not "net: tighten bad gso csum offset check in virtio_net_hdr"? ** Bug watch added: github.com/tailscale/tailscale/issues #13041 https://github.com/tailscale/tailscale/issues/13041 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2089012 Title: virtio-net slowdown and error message ens3: bad gso: type: 1, size: 1448 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2089012/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
