On 2016年11月15日 11:28, Jason Wang wrote: >On 2016年11月10日 14:19, Zhangming (James, Euler) wrote: >> On 2016年11月09日 15:14, Jason Wang wrote: >>> On 2016年11月08日 19:58, Zhangming (James, Euler) wrote: >>>> On 2016年11月08日 19:17, Jason Wang wrote: >>>> >>>>> On 2016年11月08日 19:13, Jason Wang wrote: >>>>>> Cc Michael >>>>>> >>>>>> On 2016年11月08日 16:34, Zhangming (James, Euler) wrote: >>>>>>> In container scenario, OVS is installed in the Virtual machine, >>>>>>> and all the containers connected to the OVS will communicated >>>>>>> through VXLAN encapsulation. >>>>>>> >>>>>>> By now, virtio_net does not support TSO offload for VXLAN >>>>>>> encapsulated TSO package. In this condition, the performance is >>>>>>> not good, sender is bottleneck >>>>>>> >>>>>>> I googled this scenario, but I didn’t find any information. Will >>>>>>> virtio_net support VXLAN encapsulation package TSO offload later? >>>>>>> >>>>>> Yes and for both sender and receiver. >>>>>> >>>>>>> My idea is virtio_net open encapsulated TSO offload, and >>>>>>> transport encapsulation info to TUN, TUN will parse the info and >>>>>>> build skb with encapsulation info. >>>>>>> >>>>>>> OVS or kernel on the host should be modified to support this. >>>>>>> Using this method, the TCP performance aremore than 2x as before. >>>>>>> >>>>>>> Any advice and suggestions for this idea or new idea will be >>>>>>> greatly appreciated! >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> James zhang >>>>>>> >>>>>> Sounds very good. And we may also need features bits >>>>>> (VIRTIO_NET_F_GUEST|HOST_GSO_X) for this. >>>>>> >>>>>> This is in fact one of items in networking todo list. (See >>>>>> http://www.linux-kvm.org/page/NetworkingTodo). While at it, we'd >>>>>> better support not only VXLAN but also other tunnels. >>>>> Cc Vlad who is working on extending virtio-net headers. >>>>> >>>>>> We can start with the spec work, or if you've already had some >>>>>> bits you can post them as RFC for early review. >>>>>> >>>>>> Thanks >>>> Below is my demo code >>>> Virtio_net.c >>>> static int virtnet_probe(struct virtio_device *vdev), add belows codes: >>>> if (virtio_has_feature(vdev, VIRTIO_NET_F_MRG_RXBUF) || >>>> // avoid gso segment, it should be negotiation later, >>>> because in the demo I reuse num_buffers. >>>> virtio_has_feature(vdev, VIRTIO_F_VERSION_1)) { >>>> dev->hw_enc_features |= NETIF_F_TSO; >>>> dev->hw_enc_features |= NETIF_F_ALL_CSUM; >>>> dev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL; >>>> dev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM; >>>> dev->hw_enc_features |= >>>> NETIF_F_GSO_TUNNEL_REMCSUM; >>>> >>>> dev->features |= NETIF_F_GSO_UDP_TUNNEL; >>>> dev->features |= NETIF_F_GSO_UDP_TUNNEL_CSUM; >>>> dev->features |= NETIF_F_GSO_TUNNEL_REMCSUM; >>>> } >>>> >>>> static int xmit_skb(struct send_queue *sq, struct sk_buff *skb), add >>>> below to pieces of codes >>>> >>>> if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL) >>>> hdr->hdr.gso_type |= VIRTIO_NET_HDR_GSO_TUNNEL; >>>> if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL_CSUM) >>>> hdr->hdr.gso_type |= >>>> VIRTIO_NET_HDR_GSO_TUNNEL_CSUM; >>>> if (skb_shinfo(skb)->gso_type & SKB_GSO_TUNNEL_REMCSUM) >>>> hdr->hdr.gso_type |= >>>> VIRTIO_NET_HDR_GSO_TUNNEL_REMCSUM; >>>> >>>> if (skb->encapsulation && skb_is_gso(skb)) { >>>> inner_mac_len = skb_inner_network_header(skb) - >>>> skb_inner_mac_header(skb); >>>> tnl_len = skb_inner_mac_header(skb) - >>>> skb_mac_header(skb); >>>> if ( !(inner_mac_len >> DATA_LEN_SHIFT) && !(tnl_len >> >>>> DATA_LEN_SHIFT) ) { >>>> hdr->hdr.flags |= VIRTIO_NET_HDR_F_ENCAPSULATION; >>>> hdr->num_buffers = (__virtio16)((inner_mac_len >>>> << DATA_LEN_SHIFT) | tnl_len); //we reuse num_buffers for >>>> simple , we should add extend member for later. >>>> } else >>>> hdr->num_buffers = 0; >>>> } >>>> >>>> Tun.c >>>> if (memcpy_fromiovecend((void *)&hdr, iv, offset, >>>> tun->vnet_hdr_sz)) //read header with negotiation length >>>> return -EFAULT; >>>> >>>> if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL) >>>> //set tunnel gso info >>>> skb_shinfo(skb)->gso_type |= SKB_GSO_UDP_TUNNEL; >>>> if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL_CSUM) >>>> skb_shinfo(skb)->gso_type |= >>>> SKB_GSO_UDP_TUNNEL_CSUM; >>>> if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL_REMCSUM) >>>> skb_shinfo(skb)->gso_type |= >>>> SKB_GSO_TUNNEL_REMCSUM; >>>> >>>> if (hdr.flags & VIRTIO_NET_HDR_F_ENCAPSULATION) { >>>> //read tunnel info from header and set to >>>> built skb. >>>> tnl_len = tun16_to_cpu(tun, hdr.num_buffers) & >>>> TUN_TNL_LEN_MASK; >>>> payload_mac_len = tun16_to_cpu(tun, hdr.num_buffers) >> >>>> TUN_DATA_LEN_SHIFT; >>>> mac_len = skb_network_header(skb) - skb_mac_header(skb); >>>> skb_set_inner_mac_header(skb, tnl_len - mac_len); >>>> skb_set_inner_network_header(skb, tnl_len + >>>> payload_mac_len - mac_len); >>>> skb->encapsulation = 1; >>>> } >>>> >>>> >>> Something like this, and you probably need do something more: >>> >>> - use net-next.git to generate the patch (for the latest code) >>> - add feature negotiation >>> - tun/macvtap/qemu patches for this, you can start with tun/macvtap >>> patches >>> - support for all other SKB_GSO_* types which is not supported >>> - use a new field instead of num_buffers >>> - a virtio spec patch to describe the support for encapsulation >>> offload >>> >>> Thanks >> Thank you for your advice, I will start it right now. >> >> Thanks > >Cool, one more question: while at it, I think you may want to add support for >dpdk too? > >Thanks
Do you mean that the patch should be compatible with virtio pmd, or give virtio pmd patch? Thanks
