On Nov 15, 2012, at 1:03 PM, Kyle Mestery (kmestery) <kmest...@cisco.com> wrote:
> Jesse:
>
> I modified the source port hashing for the VXLAN patch I submitted a few days
> ago,
> but I've noticed when using the upstream source port hashing routine,
> performance
> drops off by 3.5 times when using iperf between two VMs. From what I can
> tell, it
> has to be that all skbuffs coming into the VXLAN tunnel have not already had
> their
> rxhash set, and this function is what's killing performance. Let me share the
> details:
>
I think I figured this out. The upstream source port selection algorithm is
exploding flows
in the fast path. Here are iperf runs with both and subsequent "ovs-dpctl
dump-flows"
commands for comparison. The first one is with the upstream version, the second
is
with the one in my patch. Note that I just piped "ovs-dpctl dump-flows" into wc
to
summarize the flow count.
Upstream verison:
[root@linux-br ~]# iperf -c 10.1.2.14 && ovs-dpctl dump-flows | wc
------------------------------------------------------------
Client connecting to 10.1.2.14, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.2.13 port 60721 connected with 10.1.2.14 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 235 MBytes 197 Mbits/sec
1504 7523 331768
[root@linux-br ~]#
Version in my patch:
[root@linux-br ~]# iperf -c 10.1.2.14 && ovs-dpctl dump-flows | wc
------------------------------------------------------------
Client connecting to 10.1.2.14, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.2.13 port 58676 connected with 10.1.2.14 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 850 MBytes 713 Mbits/sec
16 83 3789
[root@linux-br ~]#
Looking into this more closely now.
Thanks,
Kyle
> If I use this source port hashing function and call it each time
> build_header() is called,
> performance runs at the below:
>
> static u16 get_src_port(const struct sk_buff *skb,
> const struct tnl_mutable_config *mutable)
> {
> unsigned int range = (VXLAN_SRC_PORT_MAX - VXLAN_SRC_PORT_MIN) + 1;
> u32 hash;
>
> hash = skb_get_rxhash(skb);
> if (!hash)
> hash = jhash(skb->data, 2 * ETH_ALEN,
> (__force u32) skb->protocol);
>
> return (((u64) hash * range) >> 32) + VXLAN_SRC_PORT_MIN;
> }
>
> [root@linux-br ~]# iperf -c 10.1.2.14
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
