On 8/13/25 10:31 PM, Cristian Marussi wrote: > On Wed, Aug 13, 2025 at 09:51:30PM +0200, Paolo Abeni wrote: >> On 8/13/25 8:09 PM, Cristian Marussi wrote: >>> On Wed, Aug 13, 2025 at 03:07:34PM +0100, Cristian Marussi wrote: >>>> Not sure if it has been already reported but in a kvmtool/guest setup >>>> moving >>>> the guest kernel from v6.16 to v6.17-rc1 I completely lost host-guest >>>> network >>>> functionality....in a very funny way, though, I'd say... >>>> >>>> In fact NO error is apparently reported in the guest kernel log and the >>>> interfaces seems perfectly up an running both sides, but looking at the >>>> host/guest interfaces you can see that ALL received frames are indeed >>>> dropped: >>>> >>>> >>>> enp0s1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 >>>> .... >>>> ether 02:15:15:15:15:15 txqueuelen 1000 (Ethernet) >>>> <<<<<<<<<<<<<<<< >>>> RX packets 125 bytes 17948 (17.5 KiB) >>>> RX errors 0 dropped 125 overruns 0 frame 0 >>>> TX packets 1207 bytes 51182 (49.9 KiB) >>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>>> >>>> >>>> ...on the host same..(taken later on...) >>>> >>>> tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 >>>> inet 192.168.33.1 netmask 255.255.255.0 broadcast 192.168.33.255 >>>> ether 8a:10:f6:df:a1:70 txqueuelen 1000 (Ethernet) >>>> RX packets 804 bytes 43904 (42.8 KiB) >>>> RX errors 0 dropped 804 overruns 0 frame 0 >>>> TX packets 101 bytes 14408 (14.0 KiB) >>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>>> >>>> ....and for a good reason, apparently, since sniffing around on the Host >>>> TAP >>>> interface I can see a never ending stream of: >>>> >>>> $ sudo tcpdump -i tap0 >>>> listening on tap0, link-type EN10MB (Ethernet), snapshot length 262144 >>>> bytes >>>> 22:40:42.309158 00:00:00:00:00:00 (oui Ethernet) > 00:00:00:00:00:00 (oui >>>> Ethernet), ethertype Unknown (0xffff), length 54: >>>> 0x0000: ffff ffff 0215 1515 1515 0806 0001 0800 ................ >>>> <<<<<<<<<<<<< >>>> 0x0010: 0604 0001 0215 1515 1515 c0a8 2102 0000 ............!... >>>> 0x0020: 0000 0000 c0a8 2101 ......!. >>>> >>>> ... DST/SRC Macs are just all zeros WHILE in the payload you can spot my >>>> guest >>>> SRC mac address 0215 1515 1515 :P >>>> >>> >>> I bisected this regression to: >>> >>> 56a06bd40fab64448aa6b84aa06b3dc470c1254a is the first bad commit >>> >>> commit 56a06bd40fab64448aa6b84aa06b3dc470c1254a >>> Author: Paolo Abeni <pab...@redhat.com> >>> Date: Tue Jul 8 17:55:02 2025 +0200 >>> >>> virtio_net: enable gso over UDP tunnel support. >>> >>> If the related virtio feature is set, enable transmission and reception >>> of gso over UDP tunnel packets. >>> >>> Most of the work is done by the previously introduced helper, just need >>> to determine the UDP tunnel features inside the virtio_net_hdr and >>> update accordingly the virtio net hdr size. >>> >>> Acked-by: Jason Wang <jasow...@redhat.com> >>> Signed-off-by: Paolo Abeni <pab...@redhat.com> >>> >>> drivers/net/virtio_net.c | 85 >>> ++++++++++++++++++++++++++++++++++++------------ >>> >>> Reverting this commit on top of v6.17-rc1 solves for me and network works >>> fine again. >> >> Thanks for reporting, I was not aware yet of this regression. >> > > Hi Paolo, > > I spotted this issue on my setup this morning. > >> Apparently there is mismatch between the negotiated features (that >> determinate the virtio net header size) and the actually virtio header >> used by both the guest and the host, which is totally unexpected. >> > > Have the UAPI headers been updated ? ... I could not see any change... > (not that I am very familiar with virtio core :D)
The virtio feature space has been extended from 64 bits to 128 and the blamed commit is the first using the 'extended' virtio features negotiation. But, modulo bugs, the extended space should be used only if the hypervisors exposes the relevant capabilities. kvmtool does not do that. >> Could you please share the host kernel version, the arch and the kvmtool >> version? Also I understand you only upgraded the guest kernel, without >> any other setup changes, am I correct? >> > > The arch is arm64, I could reproduce on a Raspberry Pi5 with host v6.6 and on > an Apple M1 (running Linux natively) with a v6.12. > > Both host kernels are from the original distros unchanged (Debian and Fedora) > > In both cases I compiled from scratch (make clean && make) kvmtool on the > respective arm64 Host using the current top of tree from origin/master at: > > https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/log/ > > commit ba6830eec0f5 ("vfio: include libgen.h (for musl compatibility)") > > Run kvmtoool baer simple with > > lkvm run -c 4 -m 4G -k $IMAGE_DEF --network virtio -d $ROOTFS_DEF -p > "earlycon loglevel=8" > > All was fine till v6.16, and it is my usual setup and update procedure that > never failed really > as long as I update kvmtool in sync. > Not sure if something is still missing from kvmtool master repo for v6.17 but > it seems reasonably > updated (July25). > > I wonder if no one else had this issue on arm64/kvmtool....that would made my > 'regression' > suspiciously tied tomy setup.. Thanks for the detailed reproduction steps. I agree it should be some bad iteration with the arm arch and/or kvmtool. I hope an arm system is not required to reproduce the issue, because I don't have any of them handy. I should be able to dig into this tomorrow afternoon my time. Cheers, Paolo
