Since the vxlan UDP header checksum is 0, udp_tunnel_gro_complete (called via vxlan_gro_complete) is setting SKB_GSO_UDP_TUNNEL in skb_shinfo(skb)->gso_type. Later when bridge interface tries to forward this packet to tap interface (br_dev_queue_push_xmit -> __dev_queue_xmit -> validate_xmit_skb) netif_needs_gso will succeed (skb_gso_ok returns 0) resulting in packet segmentation (which was earlier aggregated) as tap interface doesn't have this feature. Is there a way to enable tx-udp_tnl-segmentation for tap interface?
# ethtool -K tap0 tx-udp_tnl-segmentation on Could not change any device features # ethtool -k tap0 | grep -i tnl tx-udp_tnl-segmentation: off [fixed] On Tue, Jun 30, 2015 at 12:24 PM, Santosh R <skrasta...@gmail.com> wrote: > > On Mon, Jun 29, 2015 at 9:24 PM, Santosh R <skrasta...@gmail.com> wrote: > >> >> On Mon, Jun 29, 2015 at 9:04 PM, Vlad Yasevich <vyase...@redhat.com> >> wrote: >> >>> On 06/29/2015 01:46 AM, Santosh R wrote: >>> > All, >>> > >>> > I am testing VxLAN performance in VM. For this I am using below >>> command >>> > to bring up the VM. >>> > # qemu-system-x86_64 -m 4096 -smp 4 -boot c -device >>> > virtio-net-pci,netdev=tap0,mac=00:11:22:33:44:55 -netdev >>> > tap,id=tap0,script=no,vhost=on -drive file=/root/vdisk_rhel65.img & >>> > >>> > This is resulting in lower throughput in VM. On looking further I found >>> > that GRO is not happing on the tap interface. >>> > Using tcpdump I could see GRO happeing in physcial nic, vxlan and >>> bridge >>> > interface. but _not_ on tap interface. >>> >>> I don't think this is GRO. GRO only happens at the lowest possible >>> layer. >>> For packets coming from external sources, that happens just above the nic >>> when the packet enters the host for the first time. From then on, the >>> features >>> of other devices determine if the large aggregated packet will be >>> segmented >>> again or not. >>> >>> >>> > >>> > So, I thought to use veth pair instead of the tap interface in VM. >>> What is >>> > the command to do this? >>> >>> veth pair will not help with this. you need a tap or a macvtap to have >>> VM talk to host. >>> >>> > Also, >>> > 1. Why is rx-checksumming off for tap interface? >>> >>> because tap doesn't do checksum validation. it allows lower level >>> devices or >>> VM to do so. >>> >>> > 2. Why is GRO not happening with tap interface? >>> > >>> >>> Again. I don't think it's GRO. Try capturing traffic at tap and see if >>> you get >>> large packets (more the 1 mtu long) on the tap. If not, then someone >>> else is fragmenting >>> packets and you need to find who. >>> >> [Santosh]: Thanks for the reply. >> With VxLAN, aggregated packets are seen on physical, VxLAN and bridge >> interface but _not_ on the tap interface. >> Without VxLAN, I do see aggregated (GRO) packets all the way upto VM >> (i.e. physical, bridge and tap interface). >> As you rightly pointed, looks like VxLAN packets are getting fragmented. >> Any pointers to look for? >> >> > By the way, I was referring below link for VxLAN testing and ran into the > GRO issue. > This mentions "The veth0 is suppose to be connected to the VM, while the > veth1 is connected to the Linux bridge". > https://community.mellanox.com/docs/DOC-1473 >
