Hi, ALL.
Recently, we're trying to fix the poor VXLAN throughput issue by
applying the series of patches @Ramu posted in ovs-dev mailing-list.
http://openvswitch.org/pipermail/dev/2015-August/059337.html
Our test environment are as belows.
------------------------------------------
OVS: 2.4.0
Kernel: 3.18.22
Topology:
------------------------------------------
vm1 -- bridge -- vxlan -- 10Gnic -- 10Gnic -- vxlan -- bridge -- vm2
vxlan port settings:
------------------------------------------
Port "vxlan-0acd614d"
Interface "vxlan-0acd614d"
type: vxlan
options: {csum="true", df_default="true", in_key=flow,
local_ip="10.205.97.135", out_key=flow, remote_ip="10.205.97.77"}
Patch applied:
------------------------------------------
--- a/src/net/openvswitch/vport-vxlan.c
+++ b/src/net/openvswitch/vport-vxlan.c
@@ -100,6 +100,7 @@ static struct vport *vxlan_tnl_create(const struct
vport_parms *parms)
struct nlattr *a;
u16 dst_port;
int err;
+ u32 vxlan_sock_flags = 0;
if (!options) {
err = -EINVAL;
@@ -122,7 +123,8 @@ static struct vport *vxlan_tnl_create(const struct
vport_parms *parms)
vxlan_port = vxlan_vport(vport);
strncpy(vxlan_port->name, parms->name, IFNAMSIZ);
- vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true, 0);
+ vxlan_sock_flags |= VXLAN_F_UDP_CSUM;
+ vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true,
vxlan_sock_flags);
if (IS_ERR(vs)) {
But, it seems doesn't work, the VXLAN throughput is still very low(about
1.7Gbps).
After inspecting, we found GRO is seems effective for VXLAN traffic for
we can see aggregated packets on receiver nic and the function call
graph also proves that.
- 0.60% 0.12% vhost-31236 [kernel.kallsyms] [k] tcp_gro_receive ▒
- tcp_gro_receive ▒
- 99.74% tcp4_gro_receive ▒
inet_gro_receive ▒
vxlan_gro_receive ▒
udp_gro_receive ◆
udp4_gro_receive ▒
inet_gro_receive ▒
dev_gro_receive ▒
napi_gro_receive ▒
ixgbe_clean_rx_irq ▒
ixgbe_poll ▒
net_rx_action ▒
+ __do_softirq
But, on the qvo, qvb interface, the packet is fragemented again.
@jingting seems also running into the same problem.
http://openvswitch.org/pipermail/discuss/2016-January/019965.html
Any help is really appreciated. Thank you!
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
