hi, On Fri, Dec 4, 2015 at 12:46 AM, Ihar Hrachyshka <ihrac...@redhat.com> wrote: > Hi, > > Small update on the RFE. It was approved for Mitaka, assuming we come up > with proper details upfront thru neutron-specs process. > > In the meantime, we have found more use cases for flow management among > features in development: QoS DSCP, also the new OF based firewall driver. > Both authors for those new features independently realized that agent does > not currently play nice with flows set by external code due to its graceful > restart behaviour when rules with unknown cookies are cleaned up. [The agent > uses a random session uuid() to mark rules that belong to its current run.] > > Before I proceed, full disclosure: I know almost nothing about OpenFlow > capabilities, so some pieces below may make no sense. I tried to come up > with high level model first and then try to map it to available OF features. > Please don’t hesitate to comment, I like to learn new stuff! ;) > > I am thinking lately on the use cases we collected so far. One common need > for all features that were seen to be interested in proper integration with > Open vSwitch agent is to be able to manage feature specific flows on br-int > and br-tun. There are other things that projects may need, like patch ports, > though I am still struggling with the question of whether it may be > postponed or avoided for phase 1.
i suspect port management is mandatory for many of usecases. > > There are several specific operation 'kinds' that we should cover for the > RFE: > - managing flows that modify frames in-place; > - managing flows that redirect frames. > > There are some things that should be considered to make features cooperate > with the agent and other extensions: > - feature flows should have proper priorities based on their ‘kind’ (f.e. > in-place modification probably go before redirections); > - feature flows should survive flow reset that may be triggered by the > agent; > - feature flows should survive flow reset without data plane disruption > (=they should support graceful restart: > https://review.openstack.org/#/c/182920). > > With that in mind, I see the following high level design for the flow > tables: > > - table 0 serves as a dispatcher for specific features; > - each feature gets one or more tables, one per flow ‘kind’ needed; > - for each feature table, a new flow entry is added to table 0 that would > redirect to feature specific table; the rule will be triggered only if OF > metadata is not updated inside the feature table (see the next bullet); the > rule will have priority that is defined for the ‘kind’ of the operation that > is implemented by the table it redirects to; > - each feature table will have default actions that will 1) mark OF > metadata for the frame as processed by the feature; 2) redirect back to > table 0; > - all feature specific flow rules (except dispatcher rules) belong to > feature tables; > > Now, the workflow for extensions that are interested in setting flows would > be: > - on initialize() call, extension defines feature tables it will need; it > passes the name of the feature table and the ‘kind’ of the actions it will > execute; with that, the following is initialized by the agent: 1) table 0 > dispatcher entry to redirect frames into feature table; the entry has the > priority according to the ‘kind’ of the table; 2) the actual feature table > with two default rules (update metadata and push back to table 0); > - whenever extension needs to add a new flow rule, it passes the following > into the agent: 1) table name; 2) flow specific parameters (actions, > priority, ...) "actions" here means openflow actions? passing openflow actions as parameters is not simple as it might sound because they are complex objects. esp. when we have two backends. (ovs-ofctl and native of_interface) > > Since the agent will manage setting flows for extensions, it will be able to > use the active agent cookie for all feature flows; next time the agent is > restarted, it should be able to respin extension flows with no data plane > disruption. [Note: we should make sure that on agent restart, we call to > extensions *before* we clean up stale flow rules.] > > That design will hopefully allow us to abstract interaction with flows from > extensions into management code inside the agent. It should guarantee > extensions cooperate properly assuming they properly define their priorities > thru ‘kinds’ of tables they have. > > It is also assumed that existing flow based features integrated into the > agent (dvr? anti-spoofing?) will eventually move to the new flow table > management model. > > I understand that the model does not reflect how do feature processing for > existing OF based features in the agent. It may require some smart > workarounds to allow non-disruptive migration to new flow table setup. > > It would be great to see the design bashed hard before I start to put it > into spec format. Especially if it’s not sane. :) i tend to think it's better to start from a crude experimental api (eg. give the bridge objects to extension drivers) to avoid blocking subprojects too long. > > Ihar > > > Mathieu Rohon <mathieu.ro...@gmail.com> wrote: > >> Thanks ihar! >> >> On Thu, Nov 19, 2015 at 2:32 PM, Ihar Hrachyshka <ihrac...@redhat.com> >> wrote: >> UPD: now that we have some understanding what’s needed from l2 agent >> extension mechanism to cater for interested subprojects (and now that we see >> that probably the agent in focus right now is OVS only), we may move to RFE >> step. I reported the following RFE for the feature: >> >> https://bugs.launchpad.net/neutron/+bug/1517903 >> >> It may require BP if drivers team will request one. >> >> Cheers, >> >> Ihar >> >> Ihar Hrachyshka <ihrac...@redhat.com> wrote: >> >> Reviving the thread. >> >> On the design summit session dedicated to agent and plugin extensions [1] >> the following was stated for l2 agent extensions (I appreciate if someone >> checks me on the following though): >> >> - current l2 agent extensions mechanism lacks insight into agent details >> like bridges or vlan maps; >> >> - in some cases, we don’t care about extension portability across multiple >> agents, so it’s not of concern if some of them use implementation details >> like bridges to set specific flows, or to wire up some additional ports to >> them; >> >> - that said, we still don’t want extensions to have unlimited access to >> agent details; the rationale for hard constraints on what is seen inside >> extensions is that we cannot support backwards compatibility for *all* >> possible internal attributes of an agent; instead, we should explicitly >> define where we can make an effort to provide stable API into agent details, >> and what’s, on contrary, beyond real life use cases and hence can be left to >> be broken/refactored as neutron developers see fit; this API can be agent >> specific though; >> >> - agent details that are to be passed into extensions should be driven by >> actual use cases. There were several subprojects mentioned in the session >> that are assumed to lack enough access to agent attributes to do their job >> without patching core ovs agent files. Those are: BGP-VPN, SFC, (anything >> else?) Those subprojects that are interested in extending l2 agent extension >> framework are expected to come up with a list of things missing in current >> implementation, so that neutron developers can agree on proper abstractions >> to provide missing details to extensions. For that goal, I set up a new >> etherpad to collect feedback from subprojects [2]. >> >> Once we collect use cases there and agree on agent API for extensions >> (even if per agent type), we will implement it and define as stable API, >> then pass objects that implement the API into extensions thru extension >> manager. If extensions support multiple agent types, they can still >> distinguish between which API to use based on agent type string passed into >> extension manager. >> >> I really hope we start to collect use cases early so that we have time to >> polish agent API and make it part of l2 extensions earlier in Mitaka cycle. >> >> [1]: https://etherpad.openstack.org/p/mitaka-neutron-core-extensibility >> [2]: https://etherpad.openstack.org/p/l2-agent-extensions-api-expansion >> >> Ihar >> >> Ihar Hrachyshka <ihrac...@redhat.com> wrote: >> >> On 30 Sep 2015, at 12:53, Miguel Angel Ajo <mangel...@redhat.com> wrote: >> >> >> >> Ihar Hrachyshka wrote: >> On 30 Sep 2015, at 12:08, thomas.mo...@orange.com wrote: >> >> Hi Ihar, >> >> Ihar Hrachyshka : >> Miguel Angel Ajo : >> Do you have a rough idea of what operations you may need to do? >> Right now, what bagpipe driver for networking-bgpvpn needs to interact >> with is: >> - int_br OVSBridge (read-only) >> - tun_br OVSBridge (add patch port, add flows) >> - patch_int_ofport port number (read-only) >> - local_vlan_map dict (read-only) >> - setup_entry_for_arp_reply method (called to add static ARP entries) >> Sounds very tightly coupled to OVS agent. >> Please bear in mind, the extension interface will be available from >> different agent types >> (OVS, SR-IOV, [eventually LB]), so this interface you're talking about >> could also serve as >> a translation driver for the agents (where the translation is possible), I >> totally understand >> that most extensions are specific agent bound, and we must be able to >> identify >> the agent we're serving back exactly. >> Yes, I do have this in mind, but what we've identified for now seems to be >> OVS specific. >> Indeed it does. Maybe you can try to define the needed pieces in high >> level actions, not internal objects you need to access to. Like ‘- connect >> endpoint X to Y’, ‘determine segmentation id for a network’ etc. >> I've been thinking about this, but would tend to reach the conclusion that >> the things we need to interact with are pretty hard to abstract out into >> something that would be generic across different agents. Everything we need >> to do in our case relates to how the agents use bridges and represent >> networks internally: linuxbridge has one bridge per Network, while OVS has a >> limited number of bridges playing different roles for all networks with >> internal segmentation. >> >> To look at the two things you mention: >> - "connect endpoint X to Y" : what we need to do is redirect the traffic >> destinated to the gateway of a Neutron network, to the thing that will do >> the MPLS forwarding for the right BGP VPN context (called VRF), in our case >> br-mpls (that could be done with an OVS table too) ; that action might be >> abstracted out to hide the details specific to OVS, but I'm not sure on how >> to name the destination in a way that would be agnostic to these details, >> and this is not really relevant to do until we have a relevant context in >> which the linuxbridge would pass packets to something doing MPLS forwarding >> (OVS is currently the only option we support for MPLS forwarding, and it >> does not really make sense to mix linuxbridge for Neutron L2/L3 and OVS for >> MPLS) >> - "determine segmentation id for a network": this is something really >> OVS-agent-specific, the linuxbridge agent uses multiple linux bridges, and >> does not rely on internal segmentation >> >> Completely abstracting out packet forwarding pipelines in OVS and >> linuxbridge agents would possibly allow defining an interface that agent >> extension could use without to know about anything specific to OVS or the >> linuxbridge, but I believe this is a very significant taks to tackle. >> >> If you look for a clean way to integrate with reference agents, then it’s >> something that we should try to achieve. I agree it’s not an easy thing. >> >> Just an idea: can we have a resource for traffic forwarding, similar to >> security groups? I know folks are not ok with extending security groups API >> due to compatibility reasons, so maybe fwaas is the place to experiment with >> it. >> >> Hopefully it will be acceptable to create an interface, even it exposes a >> set of methods specific to the linuxbridge agent and a set of methods >> specific to the OVS agent. That would mean that the agent extension that >> can work in both contexts (not our case yet) would check the agent type >> before using the first set or the second set. >> >> The assumption of the whole idea of l2 agent extensions is that they are >> agent agnostic. In case of QoS, we implemented a common QoS extension that >> can be plugged in any agent [1], and a set of backend drivers (atm it’s just >> sr-iov [2] and ovs [3]) that are selected based on the driver type argument >> passed into the extension manager [4][5]. Your extension could use similar >> approach to select the backend. >> >> [1]: >> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/l2/extensions/qos.py#n169 >> [2]: >> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/extension_drivers/qos_driver.py >> [3]: >> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py >> [4]: >> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#n395 >> [5]: >> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py#n155 >> >> I disagree on the agent-agnostic thing. QoS extension for SR-IOV is >> totally not agnostic for OVS or LB, in the QoS case, it's just >> accidental that OVS & LB share common bridges now due to the OVS Hybrid >> implementation that leverages linux bridge >> and iptables. >> >> Wait. The QoS extension has nothing agent backend specific. All it does is >> it receives rpc updates for tracked resources and pass them into qos >> drivers. Those latter are the bits that implement backend specific >> operations. So I am not sure why you say the extension itself is agent >> specific: any other amqp based agent in the wild can adopt the extension >> as-is, only providing a new backend to load. >> >> I agree on having a well defined interface, on which API is available to >> talking back to each agent, and it has to be common, where >> it's possible to be common. >> >> It doesn't have to be easy, but it's the way if we want a world where >> those commonalities and reusability of extensions can >> exist and not be just accidental, but it's not realistic in my opinion to >> AIM for it on every shot. I believe we should try where we can >> but we should be open to agent specific extensions. The idea of the >> extensions is that you can extend specific agents without >> being forced to have the main loop hijacked, or eventually having off tree >> code plugged into our agents. >> >> Partially, yes. The culprit here is how much the extension API should know >> about an agent. We can probably make the extension API completely extendable >> by allowing agents to pass any random kwargs into the extension manager that >> will forward them into extensions. Note that it breaks current API for >> extensions and technically breaks it (not that I know of any external >> extensions that could be affected so far). >> >> There we should add support to identify the type of agent the extension >> works with (compatibility, versioning, etc..) >> >> We already pass the type into extension manager, and that’s how we plug in >> the proper backend driver in QoS. >> >> Does this approach make sense ? >> >> -Thomas >> >> >> _________________________________________________________________________________________________________________________ >> >> Ce message et ses pieces jointes peuvent contenir des informations >> confidentielles ou privilegiees et ne doivent donc >> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez >> recu ce message par erreur, veuillez le signaler >> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages >> electroniques etant susceptibles d'alteration, >> Orange decline toute responsabilite si ce message a ete altere, deforme ou >> falsifie. Merci. >> >> This message and its attachments may contain confidential or privileged >> information that may be protected by law; >> they should not be distributed, used or copied without authorisation. >> If you have received this email in error, please notify the sender and >> delete this message and its attachments. >> As emails may be altered, Orange is not liable for messages that have been >> modified, changed or falsified. >> Thank you. >> >> Note that you should really avoid putting that ^^ kind of signature into >> your emails intended for public mailing lists. If it’s confidential, why do >> you send it to everyone? And sorry, folks will copy it without >> authorisation, for archiving and indexing reasons and whatnot. >> >> Ihar >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
