Hi Irena, My MD has to take care of admin state changes since I have no L2 agent. I think that is what Bob also alluded to. That being said, I am not doing anything specific to handle admin_state_up/down. The SR-IOV port on my device is always going to be up, for now atleast.
Thanks, Sandhya On 3/5/14 1:56 AM, "Irena Berezovsky" <ire...@mellanox.com> wrote: >Hi Robert, >Seems to me that many code lines are duplicated following your proposal. >For agent based MDs, I would prefer to inherit from >SimpleAgentMechanismDriverBase and add there verify method for >supported_pci_vendor_info. Specific MD will pass the list of supported >pci_vendor_info list. The 'try_to_bind_segment_for_agent' method will >call 'supported_pci_vendor_info', and if supported continue with binding >flow. >Maybe instead of a decorator method, it should be just an utility method? >I think that the check for supported vnic_type and pci_vendor info >support, should be done in order to see if MD should bind the port. If >the answer is Yes, no more checks are required. > >Coming back to the question I asked earlier, for non-agent MD, how would >you deal with updates after port is bound, like 'admin_state_up' changes? >I'll try to push some reference code later today. > >BR, >Irena > >-----Original Message----- >From: Robert Li (baoli) [mailto:ba...@cisco.com] >Sent: Wednesday, March 05, 2014 4:46 AM >To: Sandhya Dasu (sadasu); OpenStack Development Mailing List (not for >usage questions); Irena Berezovsky; Robert Kukura; Brian Bowen (brbowen) >Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through SRIOV >binding of ports > >Hi Sandhya, > >I agree with you except that I think that the class should inherit from >MechanismDriver. I took a crack at it, and here is what I got: > ># Copyright (c) 2014 OpenStack Foundation # All Rights Reserved. ># ># Licensed under the Apache License, Version 2.0 (the "License"); you >may ># not use this file except in compliance with the License. You may >obtain ># a copy of the License at ># ># http://www.apache.org/licenses/LICENSE-2.0 ># ># Unless required by applicable law or agreed to in writing, software ># distributed under the License is distributed on an "AS IS" BASIS, >WITHOUT ># WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See >the ># License for the specific language governing permissions and >limitations ># under the License. > >from abc import ABCMeta, abstractmethod > >import functools >import six > >from neutron.extensions import portbindings from neutron.openstack.common >import log from neutron.plugins.ml2 import driver_api as api > >LOG = log.getLogger(__name__) > > >DEFAULT_VNIC_TYPES_SUPPORTED = [portbindings.VNIC_DIRECT, > portbindings.VNIC_MACVTAP] > >def check_vnic_type_and_vendor_info(f): > @functools.wraps(f) > def wrapper(self, context): > vnic_type = context.current.get(portbindings.VNIC_TYPE, > portbindings.VNIC_NORMAL) > if vnic_type not in self.supported_vnic_types: > LOG.debug(_("%(func_name)s: skipped due to unsupported " > "vnic_type: %(vnic_type)s"), > {'func_name': f.func_name, 'vnic_type': vnic_type}) > return > > if self.supported_pci_vendor_info: > profile = context.current.get(portbindings.PROFILE, {}) > if not profile: > LOG.debug(_("%s: Missing profile in port binding"), > f.func_name) > return > pci_vendor_info = profile.get('pci_vendor_info') > if not pci_vendor_info: > LOG.debug(_("%s: Missing pci vendor info in profile"), > f.func_name) > return > if pci_vendor_info not in self.supported_pci_vendor_info: > LOG.debug(_("%(func_name)s: unsupported pci vendor " > "info: %(info)s"), > {'func_name': f.func_name, 'info': >pci_vendor_info}) > return > f(self, context) > return wrapper > >@six.add_metaclass(ABCMeta) >class SriovMechanismDriverBase(api.MechanismDriver): > """Base class for drivers that supports SR-IOV > > The SriovMechanismDriverBase provides common code for mechanism > drivers that supports SR-IOV. Such a driver may or may not require > an agent to be running on the port's host. > > MechanismDrivers that uses this base class and requires an agent must > pass the agent type to __init__(), and must implement > try_to_bind_segment_for_agent() and check_segment_for_agent(). > > MechanismDrivers that uses this base class may provide supported >vendor > information, and must provide the supported vnic types. > """ > def __init__(self, agent_type=None, supported_pci_vendor_info=[], > supported_vnic_types=DEFAULT_VNIC_TYPES_SUPPORTED): > """Initialize base class for SR-IOV capable Mechanism Drivers > > :param agent_type: Constant identifying agent type in agents_db > :param supported_pci_vendor_info: a list of "vendor_id:product_id" > :param supported_vnic_types: The binding:vnic_type values we can >bind > """ > self.supported_pci_vendor_info = supported_pci_vendor_info > self.agent_type = agent_type > self.supported_vnic_types = supported_vnic_types > > def initialize(self): > pass > > @check_vnic_type_and_vendor_info > def bind_port(self, context): > LOG.debug(_("Attempting to bind port %(port)s on " > "network %(network)s"), > {'port': context.current['id'], > 'network': context.network.current['id']}) > > if self.agent_type: > for agent in context.host_agents(self.agent_type): > LOG.debug(_("Checking agent: %s"), agent) > if agent['alive']: > for segment in context.network.network_segments: > if self.try_to_bind_segment_for_agent(context, >segment, > agent): > LOG.debug(_("Bound using segment: %s"), >segment) > return > else: > LOG.warning(_("Attempting to bind with dead agent: >%s"), > agent) > else: > for segment in context.network.network_segments: > if self.try_to_bind_segment(context, segment): > LOG.debug(_("Bound using segment: %s"), segment) > return > > def validate_port_binding(self, context): > LOG.debug(_("Validating binding for port %(port)s on " > "network %(network)s"), > {'port': context.current['id'], > 'network': context.network.current['id']}) > if self.agent_type: > for agent in context.host_agents(self.agent_type): > LOG.debug(_("Checking agent: %s"), agent) > if agent['alive'] and self.check_segment_for_agent( > context.bound_segment, agent): > LOG.debug(_("Binding valid")) > return True > LOG.warning(_("Binding invalid for port: %s"), >context.current) > return False > else: > return True > > @check_vnic_type_and_vendor_info > def unbind_port(self, context): > LOG.debug(_("Unbinding port %(port)s on " > "network %(network)s"), > {'port': context.current['id'], > 'network': context.network.current['id']}) > > @abstractmethod > def try_to_bind_segment_for_agent(self, context, segment, agent): > """Try to bind with segment for agent. > > :param context: PortContext instance describing the port > :param segment: segment dictionary describing segment to bind > :param agent: agents_db entry describing agent to bind > :returns: True iff segment has been bound for agent > > Called inside transaction during bind_port() so that derived > MechanismDrivers can use agent_db data along with built-in > knowledge of the corresponding agent's capabilities to attempt > to bind to the specified network segment for the agent. > > If the segment can be bound for the agent, this function must > call context.set_binding() with appropriate values and then > return True. Otherwise, it must return False. > """ > > @abstractmethod > def check_segment_for_agent(self, segment, agent): > """Check if segment can be bound for agent. > > :param segment: segment dictionary describing segment to bind > :param agent: agents_db entry describing agent to bind > :returns: True iff segment can be bound for agent > > Called inside transaction during validate_port_binding() so > that derived MechanismDrivers can use agent_db data along with > built-in knowledge of the corresponding agent's capabilities > to determine whether or not the specified network segment can > be bound for the agent. > """ > > @abstractmethod > def try_to_bind_segment(self, segment): > """Check if segment can be bound. > > :param segment: segment dictionary describing segment to bind > :returns: True iff segment can be bound > > Called inside transaction during bind_port() so that derived > MechanismDrivers can use database data along with built-in > knowledge to attempt to bind to the specified network segment > > If the segment can be bound, this function must > call context.set_binding() with appropriate values and then > return True. Otherwise, it must return False. > """ > > > >A SRIOV MD would inherit from it and implement >try_to_bind_segment_for_agent() and check_segment_for_agent() and >try_to_bind_segment(). If an agent is required, the first two methods >should have concrete implementation, and the third may only contain >'pass'. Otherwise, the first two are 'pass', and the third needs to be >implemented. > >In the inherited class, its __init__ method should set the supported >pci_vendor_info by either hard coding or by configuration (which is >preferred so that code doesn't need to be changed with newly supported >cards). >in the bind segment method, vif_type and vif_details should be set, and >set_binding() called. > >methods inherited from MechanismDriver, such as >create_port_precommit()/postcommit(), update_port_precommit()/postcommit >can be decorated with check_vnic_type_and_vendor_info so that they won't >be called if vnic_type and vendor_info don't match. > >Let me know what you think. > >thanks, >Robert > > > > >On 3/4/14 4:08 PM, "Sandhya Dasu (sadasu)" <sad...@cisco.com> wrote: > >>Hi, >> During today's meeting, it was decided that we would re-purpose >>Robert's >>https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov to >>take care of adding a Base class to take care of common processing for >>SR-IOV ports. >> >>This class would: >> >>1. Inherits from AgentMechanismDriverBase. >>2. Implements bind_port() where the binding:profile would be checked to >>see if the port's vnic_type is VNIC_DIRECT or VNIC_MACVTAP. >>3. Also checks to see that port belongs to vendor/product that supports >>SR-IOV. >>4. This class could be used by MDs that may or may not have a valid L2 >>agent. >>5. Implement validate_port_binding(). This will always return True for >>Mds that do not have an L2 agent. >> >>Please let me know if I left out anything. >> >>Thanks, >>Sandhya >>On 2/25/14 9:18 AM, "Sandhya Dasu (sadasu)" <sad...@cisco.com> wrote: >> >>>Hi, >>> As a follow up from today's IRC, Irena, are you looking to write >>>the below mentioned Base/Mixin class that inherits from >>>AgentMechanismDriverBase class? When you mentioned port state, were >>>you referring to the validate_port_binding() method? >>> >>>Pls clarify. >>> >>>Thanks, >>>Sandhya >>> >>>On 2/6/14 7:57 AM, "Sandhya Dasu (sadasu)" <sad...@cisco.com> wrote: >>> >>>>Hi Bob and Irena, >>>> Thanks for the clarification. Irena, I am not opposed to a >>>>SriovMechanismDriverBase/Mixin approach, but I want to first figure >>>>out how much common functionality there is. Have you already looked at >>>>this? >>>> >>>>Thanks, >>>>Sandhya >>>> >>>>On 2/5/14 1:58 AM, "Irena Berezovsky" <ire...@mellanox.com> wrote: >>>> >>>>>Please see inline my understanding >>>>> >>>>>-----Original Message----- >>>>>From: Robert Kukura [mailto:rkuk...@redhat.com] >>>>>Sent: Tuesday, February 04, 2014 11:57 PM >>>>>To: Sandhya Dasu (sadasu); OpenStack Development Mailing List (not >>>>>for usage questions); Irena Berezovsky; Robert Li (baoli); Brian >>>>>Bowen >>>>>(brbowen) >>>>>Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through SRIOV >>>>>binding of ports >>>>> >>>>>On 02/04/2014 04:35 PM, Sandhya Dasu (sadasu) wrote: >>>>>> Hi, >>>>>> I have a couple of questions for ML2 experts regarding >>>>>>support of SR-IOV ports. >>>>> >>>>>I'll try, but I think these questions might be more about how the >>>>>various SR-IOV implementations will work than about ML2 itself... >>>>> >>>>>> 1. The SR-IOV ports would not be managed by ova or linuxbridge L2 >>>>>> agents. So, how does a MD for SR-IOV ports bind/unbind its ports >>>>>> to the host? Will it just be a db update? >>>>> >>>>>I think whether or not to use an L2 agent depends on the specific >>>>>SR-IOV implementation. Some (Mellanox?) might use an L2 agent, while >>>>>others >>>>>(Cisco?) might put information in binding:vif_details that lets the >>>>>nova VIF driver take care of setting up the port without an L2 >>>>>agent. >>>>>[IrenaB] Based on VIF_Type that MD defines, and going forward with >>>>>other binding:vif_details attributes, VIFDriver should do the VIF >>>>>pluging part. >>>>>As for required networking configuration is required, it is usually >>>>>done either by L2 Agent or external Controller, depends on MD. >>>>> >>>>>> >>>>>> 2. Also, how do we handle the functionality in mech_agent.py, >>>>>> within the SR-IOV context? >>>>> >>>>>My guess is that those SR-IOV MechanismDrivers that use an L2 agent >>>>>would inherit the AgentMechanismDriverBase class if it provides >>>>>useful functionality, but any MechanismDriver implementation is free >>>>>to not use this base class if its not applicable. I'm not sure if an >>>>>SriovMechanismDriverBase (or SriovMechanismDriverMixin) class is >>>>>being planned, and how that would relate to AgentMechanismDriverBase. >>>>> >>>>>[IrenaB] Agree with Bob, and as I stated before I think there is a >>>>>need for SriovMechanismDriverBase/Mixin that provides all the >>>>>generic functionality and helper methods that are common to SRIOV >>>>>ports. >>>>>-Bob >>>>> >>>>>> >>>>>> Thanks, >>>>>> Sandhya >>>>>> >>>>>> From: Sandhya Dasu <sad...@cisco.com <mailto:sad...@cisco.com>> >>>>>> Reply-To: "OpenStack Development Mailing List (not for usage >>>>>>questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>> >>>>>> Date: Monday, February 3, 2014 3:14 PM >>>>>> To: "OpenStack Development Mailing List (not for usage questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>>, Irena Berezovsky >>>>>><ire...@mellanox.com <mailto:ire...@mellanox.com>>, "Robert Li >>>>>>(baoli)" >>>>>> <ba...@cisco.com <mailto:ba...@cisco.com>>, Robert Kukura >>>>>><rkuk...@redhat.com <mailto:rkuk...@redhat.com>>, "Brian Bowen >>>>>>(brbowen)" <brbo...@cisco.com <mailto:brbo...@cisco.com>> >>>>>> Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through >>>>>>SRIOV extra hr of discussion today >>>>>> >>>>>> Hi, >>>>>> Since, openstack-meeting-alt seems to be in use, baoli and >>>>>> myself are moving to openstack-meeting. Hopefully, Bob Kukura & >>>>>> Irena can join soon. >>>>>> >>>>>> Thanks, >>>>>> Sandhya >>>>>> >>>>>> From: Sandhya Dasu <sad...@cisco.com <mailto:sad...@cisco.com>> >>>>>> Reply-To: "OpenStack Development Mailing List (not for usage >>>>>>questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>> >>>>>> Date: Monday, February 3, 2014 1:26 PM >>>>>> To: Irena Berezovsky <ire...@mellanox.com >>>>>><mailto:ire...@mellanox.com>>, "Robert Li (baoli)" <ba...@cisco.com >>>>>><mailto:ba...@cisco.com>>, Robert Kukura <rkuk...@redhat.com >>>>>><mailto:rkuk...@redhat.com>>, "OpenStack Development Mailing List >>>>>>(not for usage questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>>, "Brian Bowen (brbowen)" >>>>>> <brbo...@cisco.com <mailto:brbo...@cisco.com>> >>>>>> Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through >>>>>>SRIOV extra hr of discussion today >>>>>> >>>>>> Hi all, >>>>>> Both openstack-meeting and openstack-meeting-alt are available >>>>>> today. Lets meet at UTC 2000 @ openstack-meeting-alt. >>>>>> >>>>>> Thanks, >>>>>> Sandhya >>>>>> >>>>>> From: Irena Berezovsky <ire...@mellanox.com >>>>>><mailto:ire...@mellanox.com>> >>>>>> Date: Monday, February 3, 2014 12:52 AM >>>>>> To: Sandhya Dasu <sad...@cisco.com <mailto:sad...@cisco.com>>, >>>>>>"Robert Li (baoli)" <ba...@cisco.com <mailto:ba...@cisco.com>>, >>>>>>Robert Kukura <rkuk...@redhat.com <mailto:rkuk...@redhat.com>>, >>>>>>"OpenStack Development Mailing List (not for usage questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>>, "Brian Bowen (brbowen)" >>>>>> <brbo...@cisco.com <mailto:brbo...@cisco.com>> >>>>>> Subject: RE: [openstack-dev] [nova][neutron] PCI pass-through >>>>>>SRIOV on Jan. 30th >>>>>> >>>>>> Hi Sandhya, >>>>>> >>>>>> Can you please elaborate how do you suggest to extend the below bp >>>>>>for SRIOV Ports managed by different Mechanism Driver? >>>>>> >>>>>> I am not biased to any specific direction here, just think we need >>>>>> common layer for managing SRIOV port at neutron, since there is a >>>>>> common pass between nova and neutron. >>>>>> >>>>>> >>>>>> >>>>>> BR, >>>>>> >>>>>> Irena >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:*Sandhya Dasu (sadasu) [mailto:sad...@cisco.com] >>>>>> *Sent:* Friday, January 31, 2014 6:46 PM >>>>>> *To:* Irena Berezovsky; Robert Li (baoli); Robert Kukura; >>>>>> OpenStack Development Mailing List (not for usage questions); >>>>>> Brian Bowen >>>>>> (brbowen) >>>>>> *Subject:* Re: [openstack-dev] [nova][neutron] PCI pass-through >>>>>> SRIOV on Jan. 30th >>>>>> >>>>>> >>>>>> >>>>>> Hi Irena, >>>>>> >>>>>> I was initially looking >>>>>> at >>>>>> >>>>>>https://blueprints.launchpad.net/neutron/+spec/ml2-typedriver-extra >>>>>>-po >>>>>>r >>>>>>t >>>>>>- >>>>>>info to take care of the extra information required to set up the >>>>>>SR-IOV port. >>>>>> When the scope of the BP was being decided, we had very little >>>>>>info about our own design so I didn't give any feedback about >>>>>>SR-IOV ports. >>>>>> But, I feel that this is the direction we should be going. Maybe >>>>>>we should target this in Juno. >>>>>> >>>>>> >>>>>> >>>>>> Introducing, */SRIOVPortProfileMixin /*would be creating yet >>>>>> another way to take care of extra port config. Let me know what you >>>>>>think. >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Sandhya >>>>>> >>>>>> >>>>>> >>>>>> *From: *Irena Berezovsky <ire...@mellanox.com >>>>>> <mailto:ire...@mellanox.com>> >>>>>> *Date: *Thursday, January 30, 2014 4:13 PM >>>>>> *To: *"Robert Li (baoli)" <ba...@cisco.com >>>>>> <mailto:ba...@cisco.com>>, Robert Kukura <rkuk...@redhat.com >>>>>> <mailto:rkuk...@redhat.com>>, Sandhya Dasu <sad...@cisco.com >>>>>> <mailto:sad...@cisco.com>>, "OpenStack Development Mailing List >>>>>>(not for usage questions)" >>>>>> <openstack-dev@lists.openstack.org >>>>>> <mailto:openstack-dev@lists.openstack.org>>, "Brian Bowen (brbowen)" >>>>>> <brbo...@cisco.com <mailto:brbo...@cisco.com>> >>>>>> *Subject: *RE: [openstack-dev] [nova][neutron] PCI pass-through >>>>>> SRIOV on Jan. 30th >>>>>> >>>>>> >>>>>> >>>>>> Robert, >>>>>> >>>>>> Thank you very much for the summary. >>>>>> >>>>>> Please, see inline >>>>>> >>>>>> >>>>>> >>>>>> *From:*Robert Li (baoli) [mailto:ba...@cisco.com] >>>>>> *Sent:* Thursday, January 30, 2014 10:45 PM >>>>>> *To:* Robert Kukura; Sandhya Dasu (sadasu); Irena Berezovsky; >>>>>> OpenStack Development Mailing List (not for usage questions); >>>>>> Brian Bowen (brbowen) >>>>>> *Subject:* [openstack-dev] [nova][neutron] PCI pass-through SRIOV >>>>>> on Jan. 30th >>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> We made a lot of progress today. We agreed that: >>>>>> >>>>>> -- vnic_type will be a top level attribute as binding:vnic_type >>>>>> >>>>>> -- BPs: >>>>>> >>>>>> * Irena's >>>>>> https://blueprints.launchpad.net/neutron/+spec/ml2-request-vnic-ty >>>>>> pe >>>>>> for binding:vnic_type >>>>>> >>>>>> * Bob to submit a BP for binding:profile in ML2. SRIOV input >>>>>>info will be encapsulated in binding:profile >>>>>> >>>>>> * Bob to submit a BP for binding:vif_details in ML2. SRIOV >>>>>>output info will be encapsulated in binding:vif_details, which may >>>>>>include other information like security parameters. For SRIOV, >>>>>>vlan_id and profileid are candidates. >>>>>> >>>>>> -- new arguments for port-create will be implicit arguments. >>>>>> Future release may make them explicit. New argument: >>>>>> --binding:vnic_type {virtio, direct, macvtap}. >>>>>> >>>>>> I think that currently we can make do without the profileid as an >>>>>> input parameter from the user. The mechanism driver will return a >>>>>> profileid in the vif output. >>>>>> >>>>>> >>>>>> >>>>>> Please correct any misstatement in above. >>>>>> >>>>>> >>>>>> >>>>>> Issues: >>>>>> >>>>>> -- do we need a common utils/driver for SRIOV generic parts to >>>>>> be used by individual Mechanism drivers that support SRIOV? More >>>>>> details on what would be included in this sriov utils/driver? I'm >>>>>> thinking that a candidate would be the helper functions to >>>>>> interpret the pci_slot, which is proposed as a string. Anything >>>>>>else in your mind? >>>>>> >>>>>> */[IrenaB] I thought on some SRIOVPortProfileMixin to handle and >>>>>> persist SRIOV port related attributes/* >>>>>> >>>>>> >>>>>> >>>>>> -- what should mechanism drivers put in binding:vif_details and >>>>>> how nova would use this information? as far as I see it from the >>>>>> code, a VIF object is created and populated based on information >>>>>> provided by neutron (from get network and get port) >>>>>> >>>>>> >>>>>> >>>>>> Questions: >>>>>> >>>>>> -- nova needs to work with both ML2 and non-ML2 plugins. For >>>>>>regular plugins, binding:vnic_type will not be set, I guess. Then >>>>>>would it be treated as a virtio type? And if a non-ML2 plugin >>>>>>wants to support SRIOV, would it need to implement vnic-type, >>>>>>binding:profile, binding:vif-details for SRIOV itself? >>>>>> >>>>>> */[IrenaB] vnic_type will be added as an additional attribute to >>>>>> binding extension. For persistency it should be added in >>>>>> PortBindingMixin for non ML2. I didn't think to cover it as part >>>>>> of >>>>>> ML2 vnic_type bp./* >>>>>> >>>>>> */For the rest attributes, need to see what Bob plans./* >>>>>> >>>>>> >>>>>> >>>>>> -- is a neutron agent making decision based on the >>>>>>binding:vif_type? >>>>>> In that case, it makes sense for binding:vnic_type not to be >>>>>> exposed to agents. >>>>>> >>>>>> */[IrenaB] vnic_type is input parameter that will eventually cause >>>>>> certain vif_type to be sent to GenericVIFDriver and create network >>>>>> interface. Neutron agents periodically scan for attached interfaces. >>>>>> For example, OVS agent will look only for OVS interfaces, so if >>>>>> SRIOV interface is created, it won't be discovered by OVS agent./* >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Robert >>>>>> >>>>> >>>> >>>> >>>>_______________________________________________ >>>>OpenStack-dev mailing list >>>>OpenStack-dev@lists.openstack.org >>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>>_______________________________________________ >>>OpenStack-dev mailing list >>>OpenStack-dev@lists.openstack.org >>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev