Currently, a Netlink operation that deletes a flow can send its response
to userspace before the flow is completely "gone", because another CPU
can see the flow just before the deletion becomes visible on that CPU
and still be processing its actions afterward.
I don't know if this is actually a scenario realistically worth worrying
about. Is the probability that it would happen, given an active flow
being deleted, 1%? .1%? .000001%? Whatever it is, the probability of
it happening in OVS is probably less, because OVS normally deletes only
flows that are idle.
Why do we care? Current userspace isn't really affected. At most, we
get flow statistics wrong by one packet (I guess multiple packets is
theoretically possible?), if and when this happens. The place where it
keeps coming up is in situations where we want userspace to be able to
associate metadata with a packet sent from kernel to user. This came up
in August regarding sflow:
http://openvswitch.org/pipermail/dev/2011-August/010384.html
Now it's coming up again regarding the OF1.2 version of OFPT_PACKET_IN.
In OF1.2, OFPT_PACKET_IN can send packet metadata (e.g. NXM_NX_REG
values) to the controller, so we have to have some way to associate
those values with the flow. One way (there are others) is to keep that
data in userspace, keyed on some unique ID, but expiration is a problem
because there's no sure way to know when packets cannot show up in
userspace for a deleted flow.
So far I've thought of the following ways for userspace to figure out
when a kernel flow is really gone:
1. Use a timer. Not really satisfying.
2. Somehow learn when a kernel grace period has elapsed. Not a
good idea since it's so bound to the particular kernel
implementation.
3. Somehow actually eliminate the problem with deleting flows,
so that when userspace receives the response to the flow
deletion we know that no more packets can go through the
flow. I don't know how to do this efficiently.
4. Have the RCU callback for flow deletion send the Netlink
broadcast message that tells userspace that the flow is gone.
The Netlink client that sent the actual deletion request
would still get a synchronous response, but the broadcast
would be delayed until the flow was really gone. I think
this might be practical, but I don't know the exact
restrictions on RCU callbacks.
Thoughts?
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
