Recent bugs[1] have highlighted a particular situation where we may handle
significant traffic in userspace via the upcall mechanism either due to flow
table changes, or when bugs in translation logic result in unexpected deletion
of datapath flows.
The basic logic is this:
* A high-throughput flow in the datapath is revalidated near the beginning of
the revalidation cycle ("dump" phase).
* Revalidator decides to delete the flow.
* The ukey for the flow is kept around to ensure that duplicate dumps from
the datapath do not double-attribute (stats + side-effects).
* Once the flow is deleted, traffic begins to upcall
* During upcall processing, a flow will only be installed if the corresponding
ukey can be set up.
* However, the old ukey is still in place until the revalidators reach the
"sweep" phase.
* As such, the packets for this flow are handled via the upcall interface
until the revalidator sweeps it.
When combined with a bug in translation logic, this may cause a kind of
oscillation to occur, with the packets flowing through the datapath, then up
into userspace, then back down, then back up. Under high load this could be
hundreds of milliseconds per cycle, rather than seeing packets primarily flow
through the datapath with occasional blips of userspace processing.
Even if there is no such bug, this may also have a larger impact than
necessary on high throughput flows when the flow table changes.
This series does some code cleanup then addresses the behaviour above by
detecting when this situation occurs, and attempting to replace the old
ukey with a new ukey, thereby allowing the flow to be installed, and to
expedite putting the flow back into the datapath. This should improve
OVS resiliency in some corner cases.
[1] http://openvswitch.org/pipermail/dev/2016-August/078855.html
Joe Stringer (4):
upcall: Reuse flow_put initializer.
upcall: Only init flow_put if ukey is installed.
upcall: Track ukey states.
upcall: Replace ukeys for deleted flows.
ofproto/ofproto-dpif-upcall.c | 208 +++++++++++++++++++++++++++---------------
1 file changed, 136 insertions(+), 72 deletions(-)
--
2.9.3
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
