In general we agree that it would be good to extend the OVS bundle support to cover groups and meters in accordance with the OF 1.4 specification and that it would be helpful to also make it available as extension to OF 1.3 according to ONF #230.
However, we still have doubts whether the bundle mechanism is the right tool to solve the resync problem between controller and OVS. The controller needs to resync state with a switch after a temporary connection loss or a local restart. In such situations the controller knows the required flow state of the switch (as it is the master of this state) but it cannot be certain to have an accurate view of the current state on the switch: the switch may have restarted and lost state, flows may have timed out, the switch may have connected to a different controller in between etc. A controller can of course download the entire required state to the affected switch, but it also needs a way to clear any stale state on the switch (without being aware of it). Doing that efficiently and with non-stop forwarding is what our resync proposal is about. It is not about doing an atomic switch between current (unknown) state and new state. A controller cannot reliably update the current flow state incrementally unless it knows the current state for sure. The canonical OF method for a controller to perform an incremental resync of flow state on the switch is therefore to 1. Buffer all concurrent updates to a switch 2. Query all flows/groups/meters from the switch 3. Perform a comparison between status quo and required flow state 4. Generate delta flow/group/meter add/mod/delete messages to adjust the status quo to the required state (or, if more appropriate, delete all and download entire state from scratch) 5. Forward any buffered commands to the switch No controller we are aware of has actually implemented such a scheme for state resync. It is both complicated and inefficient. With the bundle mechanism, a controller could indeed resync flow state simply by first deleting all state and then download the entire state, all within a single big bundle to trigger an atomic switch between old and new state. (It is not possible to create an incremental update bundle without the complex procedure above.) So the memory overhead and the potentially long bundle commit processing time remain as main obstacles in OVS to using the bundle mechanism for resync. The bundle mechanism was really designed for atomic updates of the OF pipeline. Our resync mechanism, in contrast, doesn't need/have the atomicity. It refreshes the flows/groups/meters in place and leave the deletion of stale state until after the completion. And, as state download during resync happens with normal OF request messages, there is no need for special handling of asynchronous state updates during resync in the controller. I cannot really see how we could enhance the bundle mechanism in OVS to provide similar characteristics, as OF bundles must always be controller atomic, even if they were not packet-atomic. But I am very interested in your ideas and suggestions. BR, Jan BTW: The actual implementation of the OF bundle protocol [Jan] support in ODL is just work that needs doing. No problem there (apart from timing). The more difficult aspect would be to integrate the state reconciliation logic in ODL's Forwarding Rule Manger (FRM) with the OF bundle mechanism to make sure that any asynchronous (application driven) flow/group/meter updates during a resync procedure are either buffered or included in the resync bundle. I am confident, however, that this could be solved in ODL if it was agreed that the OF bundle mechanism is indeed the tool to use for state resync. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
