Hi Jesse, Ben, I think I might have figured out performance issue with mpls/qinq and tcp offload mentioned earlier. The driver(intel e1000e driver) checks skb->protocol to be either ETH_P_IP/ETH_P_IPV6 and sets the flag in DMA descriptors based on layer-4. This logic is basically used in checksum offloading. In mpls and qinq, I used to update skb->protocol to respective eth_types which then confused the driver and didn't see flags correctly. I modified the mpls/qinq not to update skb->protocol(just one line change) and things look pretty good. The performance numbers I am getting right now are
VLAN + MPLS tagged packet - 879 Mbits/sec, uses kernel offload code MPLS single tag - 934 Mbits/sec, uses OVS offload code MPLS double tag - 934 Mbits/sec, uses OVS offload code QinQ (0x8100 outer tag) - 920 Mbits/sec, uses OVS offload code QInQ (0x88a8 outer tag) - 932 Mbits/sec, uses OVS offload code All testing were done on Ubuntu 11.04 and 12.04 physical systems connected via ethernet switch and iperf. For single tag (either vlan or mpls) no mtu change is required. For double tag(vlan + mpls, mpls 2 tags, qinq), I increased mtu on receiving side to 1504. Thought of keeping you guys in loop, please let me know if its the right approach? Thanks, Ravi On Thu, May 31, 2012 at 10:16 AM, ravi kerur <rke...@gmail.com> wrote: > Hi Jesse, > > Attached latest code diffs. It includes changes to handle offload as > suggested + incorporates Ben's code review comments. > > Hi Ben, > > I have generated single patch for OF 1.1 and 1.3 changes assuming it > would be easier for code-review but as I suggested earlier, I will > probably have 2 separate commits. Please let me know if you want me to > separate them for code review as well. > > Also QinQ patch is attached just in case someone needs it. > > Thanks, > Ravi > > On Tue, May 29, 2012 at 3:47 PM, ravi kerur <rke...@gmail.com> wrote: >> Thanks Jesse, I will check the code in vport-netdev.c file and get >> back with the diffs. I did modify the code to get earlier version >> working. Anyways will get back to you. >> >> On Tue, May 29, 2012 at 3:32 PM, Jesse Gross <je...@nicira.com> wrote: >>> On Tue, May 29, 2012 at 1:32 PM, ravi kerur <rke...@gmail.com> wrote: >>>> Thanks Jesse. What I have tested is 2 Ubuntu 11.04 and 12.04 physical >>>> systems connected via a ethernet switch. I have tested scp downloads, >>>> iperf + netperf. Flows are setnsup such that MPLS tag is pushed for >>>> every outgoing TCP segments. I do see via tcpdump that a jumbo packet >>>> (~15K) is sent out by scp and NIC chops the jumbo packet based on the >>>> MTU of the NIC before sending out. On the receive side, I do see >>>> around 10 1500 byte packets being received(having same IPID). I don't >>>> think LRO/GRO is taking place on the receive side though. This I have >>>> tested successfully. >>>> >>>> The reason I mentioned about NIC was because I was seeing messages >>>> "checksum partial, protocol=0x8847/0x88a8..." from the driver. The >>>> driver complains about this when it is sending out a template about >>>> checksums offsets and other things to NIC and has to be modified to >>>> handle new eth_types. >>> >>> Drivers need to report to the network stack what types of offloading >>> they support. Since there's no current support for MPLS, no driver >>> reports this capability and should not receive MPLS packets that need >>> to be offloaded. This means that even if it works the behavior is not >>> correct. >>> >>>> I have looked at vport-netdev.c and tunnel.c files and modified code >>>> to handle MPLS and QinQ tags. Can you please point me which function >>>> to be specific for software offload emulation. I will modify if I have >>>> not already handled it. >>> >>> Generally speaking the emulation code is handled by skb_gso_segment() >>> in dev.c in the kernel code outside of OVS. This should mostly work >>> except that it needs to be able to detect that MPLS requires >>> emulation. This will be the easiest part to get working and is the >>> best place to start. However, in order for this code to work on any >>> kernel before your changes get integrated (i.e. Linux 3.6 at the >>> earliest) you'll have to emulate it in OVS as well, like we do for >>> vlans in vport-netdev.c. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
