On 1/22/14 3:23 PM, "Ben Pfaff" <b...@nicira.com> wrote:
>On Wed, Jan 22, 2014 at 08:17:05PM +0000, McGarvey, Kevin wrote: >> >> >> On 1/22/14 12:44 PM, "Ben Pfaff" <b...@nicira.com> wrote: >> >> >On Wed, Jan 22, 2014 at 09:39:14AM -0800, Ben Pfaff wrote: >> >> On Wed, Jan 22, 2014 at 05:35:40PM +0000, McGarvey, Kevin wrote: >> >> > >> >> > >> >> > On 1/21/14 6:17 PM, "Ben Pfaff" <b...@nicira.com> wrote: >> >> > >I'd expect a dramatic drop in CPU consumption in that case. There >> >>are >> >> > >a few special cases where the upgrade wouldn't help. One is if >> >> > >in-band control is in use, another is if NetFlow is turned on, a >> >>third >> >> > >is if LACP bonds with L4 port based hashing are turned on, and >>there >> >> > >are probably a few others that don't come to mind immediately. >> >> > >> >> > I plan to rerun the test to rule out some mistake on my part. >> >> > >> >> > Could you provide more information about the nature of the change >> >>made in >> >> > 1.11 that improves performance for this type of traffic? Is the >> >>kernel >> >> > module able to forward UDP DNS packets without sending them to >> >>userspace, >> >> > or was it an optimization of the userspace processing? What >>roughly >> >>is >> >> > the level of performance I should see? >> >> >> >> In 1.11 and later, for simple OpenFlow tables (I don't think you >> >> mentioned whether you are using a controller or which one), Open >> >> vSwitch can set up only a single kernel flow that covers many >>possible >> >> flows, for example all possible UDP destination ports, rather than >> >> setting up an individual kernel flow for each UDP packet. When that >> >> works, it eliminates most of the kernel/userspace traffic, improving >> >> performance. Version 2.0 is better at analyzing OpenFlow flow tables >> >> to see when this is possible, so it can better take advantage of the >> >> ability. >> > >> >I see that I didn't answer your question about performance. >> > >> >When this optimization kicks in fully, I guess that the performance >> >should be about the same as for traffic with long flows (like the >> >netperf TCP_STREAM test, for example) in terms of packets per second. >> >> Thanks. This is encouraging. The only question is why isn't the >> optimization kicking in? >> >> >> I repeated the test, and under a load of 10K DNS requests/responses per >> second ovs-vswitchd is using 82% of a core. >> >> I wasn't sure whether in-band control was on or off by default, so I >> disabled it with the command below and restarted openvswitch, but the >>cpu >> consumption didn't change: >> >> ovs-vsctl set bridge <bridge> other-config:disable-in-band=true >> >> I did not set up the configuration, but as far as I can tell Netflow is >> not turned on. The output of 'ovsdb-tool -show-log | grep -i netflow' >>is >> empty. >> >> There are no bonded interfaces. The 2 NICs used for DNS traffic are >> associated with separate bridges. >> >> We are not using a controller. >> >> In your response you mentioned that for simple OpenFlow tables Open >> vSwitch can set up a single kernel flow that covers many possible flows. >> I think this is exactly what I need. Do I need to add a flow using >> ovs-ofctl? > >No. With the settings you describe, it should kick in automatically. > >Here is an experiment that might help. Take one of the flows that >"ovs-dpctl dump-flows" prints, then feed that flow back into >"ovs-appctl ofproto/trace", and show us the results. (You might have >to spend a few minutes reading the ovs-vswitchd manpage to learn the >ofproto/trace syntax, if you don't already know it.) Below is the ofproto/trace output for an inbound request to bridge brsvr2. One more piece of information is that the packets are going through a load balancer. ovs-appctl ofproto/trace brsvr2 "in_port(4),eth(src=00:1f:26:3d:e8:00,dst=52:54:00:a1:27:1e),eth_type(0x810 0),vlan(vid=138,pcp=0),encap(eth_type(0x0800),ipv4(src=40.90.75.212,dst=10. 63.101.234,proto=17,tos=0,ttl=60,frag=no),udp(src=26011,dst=53))" Bridge: brsvr2 Flow: udp,metadata=0,in_port=1,dl_vlan=138,dl_vlan_pcp=0,dl_src=00:1f:26:3d:e8:00 ,dl_dst=52:54:00:a1:27:1e,nw_src=40.90.75.212,nw_dst=10.63.101.234,nw_tos=0 ,nw_ecn=0,nw_ttl=60,tp_src=26011,tp_dst=53 Rule: table=0 cookie=0 priority=0 OpenFlow actions=NORMAL forwarding to learned port Final flow: unchanged Relevant fields: skb_priority=0,ip,in_port=1,dl_vlan=138,dl_src=00:1f:26:3d:e8:00,dl_dst=52: 54:00:a1:27:1e,nw_frag=no Datapath actions: 9 And here is the output for a response from the VM: ovs-appctl ofproto/trace brsvr2 "in_port(9),eth(src=52:54:00:a1:27:1e,dst=90:e2:ba:26:89:45),eth_type(0x810 0),vlan(vid=138,pcp=0),encap(eth_type(0x0800),ipv4(src=10.63.101.234,dst=40 .90.85.153,proto=17,tos=0,ttl=64,frag=no),udp(src=53,dst=26292))" Bridge: brsvr2 Flow: udp,metadata=0,in_port=2,dl_vlan=138,dl_vlan_pcp=0,dl_src=52:54:00:a1:27:1e ,dl_dst=90:e2:ba:26:89:45,nw_src=10.63.101.234,nw_dst=40.90.85.153,nw_tos=0 ,nw_ecn=0,nw_ttl=64,tp_src=53,tp_dst=26292 Rule: table=0 cookie=0 priority=0 OpenFlow actions=NORMAL forwarding to learned port Final flow: unchanged Relevant fields: skb_priority=0,ip,in_port=2,dl_vlan=138,dl_src=52:54:00:a1:27:1e,dl_dst=90: e2:ba:26:89:45,nw_frag=no Datapath actions: 4 _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
