Jesse,
I ran an iperf test (4 client server pairs) with and without the stats and
there was no noticeable difference in the measured bandwidth. However as I
increase the number of connections from 1 to 4 I do see the spin lock
overhead for this call stack (as a percent of total spin lock overhead on
the system) go up from 6% to 16%. I have 2 numa nodes on my system and 32
cores. But yes it is not affecting bandwidth.
- 16.49% ovs_flow_stats_update
ovs_dp_process_packet_with_key
ovs_dp_process_received_packet
ovs_vport_receive
netdev_frame_hook
__netif_receive_skb_core
__netif_receive_skb
recirc_id(0),skb_priority(0),in_port(2),eth(src=90:e2:ba:59:1b:c4,dst=90:e2:ba:61:e4:54),eth_type(0x0800),ipv4(src=
12.1.1.2/0.0.0.0,dst=12.1.1.1/0.0.0.0,proto=6/0,tos=0/0,ttl=64/0,frag=no/0xff),
packets:0, bytes:0, used:never, actions:3
recirc_id(0),skb_priority(0),in_port(3),eth(src=90:e2:ba:61:e4:54,dst=90:e2:ba:59:1b:c4),eth_type(0x0800),ipv4(src=
12.1.1.1/0.0.0.0,dst=12.1.1.2/0.0.0.0,proto=6/0,tos=0/0,ttl=64/0,frag=no/0xff),
packets:0, bytes:0, used:never, actions:2
[ 3] 0.0-300.0 sec 83.5 GBytes 2.39 Gbits/sec
[ 3] 0.0-300.0 sec 80.1 GBytes 2.29 Gbits/sec
[ 3] 0.0-300.0 sec 82.3 GBytes 2.36 Gbits/sec
[ 3] 0.0-300.0 sec 80.3 GBytes 2.30 Gbits/sec
Thanks.
On Sun, May 25, 2014 at 10:34 AM, Jesse Gross <je...@nicira.com> wrote:
> On Sat, May 24, 2014 at 6:06 PM, Madhu Challa <cha...@noironetworks.com>
> wrote:
> > Hi Jesse,
> >
> > Thanks for the clarifications.
> >
> >
> > On Fri, May 23, 2014 at 9:52 AM, Jesse Gross <je...@nicira.com> wrote:
> >>
> >> On Thu, May 22, 2014 at 5:02 PM, Madhu Challa <cha...@noironetworks.com
> >
> >> wrote:
> >> > I am looking at adding support for On-demand flow counters and figured
> >> > one
> >> > of the benefits of doing this is reduced lock contention in
> >> > ovs_flow_stats_update.
> >> >
> >> > If my understanding is correct, this involves processing flags
> >> > OFPFF13_NO_PKT_COUNTS and OFPFF13_NO_BYT_COUNTS and communicating them
> >> > so
> >> > when the kernel flows are created / updated we somehow pass this info
> >> > along
> >> > so the stats do not get updated for such flows.
> >>
> >> It's not clear that this is really the case. OVS still needs to
> >> maintain stats in order to age out idle flows, etc. even if they
> >> aren't used by the controller.
> >
> >
> > I was making this assumption based on the comments in the TODO list.
> >
> > " * On-demand flow counters. I think this might be a real optimization in
> > some cases for the software switch.
> > [optional for OF1.3+]"
> >
> > I would guess the last used field is the one needed to age out the flows,
> > but if we also have to increment the counters we would need the lock and
> > there would not be much savings I can think of.
>
> My guess is that even just maintaining the last used field would be
> about the same. One other data point is that moving from per-CPU to
> per-NUMA stats didn't appear to have much effect on performance even
> for flows on different cores in the same socket, which would indicate
> that maybe there isn't much overhead here after all.
>
> It would be fairly easy to test out the potential performance gain by
> simply not updating the stats in the kernel at all and see what
> happens. I would try that before doing the work to plumb everything
> through.
>
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
