I know that vif_type is binding_failed on a multinode setup and I also know why it happens.
As for interface-attach I got it work for sriov ports and even verified it works inside the instance. The trick was to specify profile with pci_slot and pci_vendor_info during port create. In case any one else wants to do this. Thank you, Ageeleshwar K On Thu, Feb 5, 2015 at 12:19 PM, Irena Berezovsky <[email protected]> wrote: > Hi Akilesh, > Please see my responses inline. > Hope this help, > > BR, > Irena > > On Thu, Feb 5, 2015 at 6:14 AM, Akilesh K <[email protected]> wrote: > >> Hi Irena, >> >> Issue 1 - I agree. You are correct. >> >> Issue 2 >> The behavior you outlined >> 1. When port is created with vnic_type=direct, the vif_type is >> 'unbound'. The pci_vendor_info will be available during port update when >> 'nova boot' command is invoked and PCI device is allocated. >> This happens when the controller and compute are on the same host. Not >> when they are on the different host. On a multiserver setup vif_type is set >> to binging_failed during port create. >> >> This is strange, since port-create operation is pure neutron API call and > it should not differ whether you are in the multiserver or all-in-one setup. > > Second is i am not doing nova boot. Instead I am doing nova >> interface-attach. In this case the pci_vendor_info is not updated by anyone >> but me. and pci_slot is also not populated. >> >> interface-attach is currently not supported for SR-IOV ports. There is a > proposed blueprint to support this: > https://review.openstack.org/#/c/139910/. > So for now, the only option to provide PCI passthrough vNIC is according > to what is described in the previously referenced wiki page: create neutron > port with vnic_type= direct and then 'nova boot' with pre-created port. > > Do you still think this is correct ? > >> >> >> >> On Wed, Feb 4, 2015 at 8:08 PM, Irena Berezovsky <[email protected]> >> wrote: >> >>> Hi Akilesh, >>> please see inline >>> >>> On Wed, Feb 4, 2015 at 11:32 AM, Akilesh K <[email protected]> >>> wrote: >>> >>>> Hi, >>>> Issue 1: >>>> I do not understand what you mean. I did specify the physical_network. >>>> What I am trying to say is some physical networks exists only on the >>>> compute node and not on the network node. We are unable to create a network >>>> on those physnets. The work around was to fake their existance on the >>>> network node too. Which I believe is the wrong way to do. >>>> >>>> Every physical network should be defined at the Controller node, >>> including range of segmentation ids (i.e. vlan ids) available for >>> allocation. >>> When virtual network is created, you should verify that it has >>> associated network type and segmentation id (assuming you are using >>> provider network extension). >>> >>>> >>>> Issue2: >>>> I looked directly into the code after looking at the logs. >>>> >>>> 1. What neutron (sriov mech driver ) is doing is loading the default >>>> list of 'supported_pci_vendor_devs' , then it picks up the >>>> profile->pci_vendor_info from the port defenition we sent in the port >>>> create request and checks if it is supported. If not it says >>>> 'binding_failed' >>>> >>> When port is created with vnic_type=direct, the vif_type is 'unbound'. >>> The pci_vendor_info will be available during port update when 'nova boot' >>> command is invoked and PCI device is allocated. >>> >>> >>>> >>>> I am fine with this >>>> >>>> 2. Then when I attach the created port to a host nova's vif driver >>>> (hv_veb) is looking for profile->pci_slot in the context of the port that >>>> was supplied and fails to attach to the instance if it is not present. >>>> >>>> nova vif driver receives profile->pci_slot from neutron, but it was >>> actually filed earlier by nova during port-update. >>> >>>> this is what I think should be done by neutron itself. neutron's sriov >>>> mech driver should have updated the port with the pci_slot details when the >>>> port got created. and this does happen on a single machine install. We need >>>> to find why it does not happen on a multi node install, possibly because >>>> the mech driver is not running on the host with sriov devices and fix it. >>>> >>>> I suggest to follow >>> https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking >>> instructions, >>> this should work for you. >>> >>> I hope you guys can understand what I mean. >>> >>>> >>>> Thank you, >>>> Ageeleshwar K >>>> >>>> >>>> On Wed, Feb 4, 2015 at 2:49 PM, Itzik Brown <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Issue 1; >>>>> You must specify the physical networks. >>>>> Please look at: >>>>> https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking >>>>> >>>>> Issue 2: >>>>> AFAIK the agent is supported by only one vendor. >>>>> Can you please look for errors in Neutron's log? >>>>> >>>>> Thanks, >>>>> Itzik >>>>> >>>>> On 02/04/2015 09:12 AM, Akilesh K wrote: >>>>> >>>>> Hi, >>>>> I found two issues with the way neutron behaves on a multi server >>>>> install. I got it to work but I do not this this is the right way to do >>>>> it. >>>>> It might be a bug we might want to fix and for which I could volunteer. >>>>> >>>>> Setup - Multiserver juno on ubuntu. >>>>> >>>>> Machine 1 - Controller >>>>> All api servers , l3, dhcp and ovs agent >>>>> >>>>> Machine 2 - Compute >>>>> nova compute, neutron-ovs-agent, neutron sriov agent. >>>>> >>>>> >>>>> Issue 1: >>>>> >>>>> Controller node has physnets 'External', 'Internal' configured in ml2 >>>>> >>>>> Compute node has physnets 'Internal', 'Physnet1', 'Physnet2' >>>>> configured in ml2 >>>>> >>>>> When I do neutron net-create --provider:physicalnetwork Physnet1, It >>>>> complains that 'Physnet1' is not available. >>>>> >>>>> Offcourse its not available on the controller but is available on the >>>>> compute node and there is no way to tell neutron to host that network on >>>>> compute node alone >>>>> >>>>> Work around >>>>> I had to include 'Physnet1' in the controller node also to get it to >>>>> work, except that there is not bridge mapings for this physnet. >>>>> >>>>> >>>>> Issue 2: >>>>> >>>>> This is related to sriov agent. This agent is configured only on the >>>>> compute node as that node alone has supported devices. >>>>> >>>>> When I do a port create --binding:vnic_type direct --binding:host_id >>>>> <compute node> The port is created but with binding:vif_type: >>>>> *'binding-failed'*. and naturally I could not attach it to any >>>>> instance. >>>>> >>>>> I looked at the code and figured out that neutron api is expecting >>>>> binding:profile also in the format >>>>> {"pci_slot": "0000:03:10.1", "pci_vendor_info": "8086:10ed"} >>>>> >>>>> Is this how it should be. Because on a single machine install I did >>>>> not have to do this. However on a multiserver I had to even give the pci >>>>> address is the exact format to get it to work. >>>>> >>>>> I have a serious feeling that this could be lot simpler if neutron >>>>> could take care of finding the details in a smart way rather than relying >>>>> on the administrator to find which device is available and configure it. >>>>> >>>>> >>>>> Note: >>>>> 1. If I can get some expert advice I can fix both these. >>>>> 2. I am not sure if this question should rather be sent to >>>>> openstack-dev group. Let me know. >>>>> >>>>> >>>>> Thank you, >>>>> Ageeleshwar K >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mailing list: >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>> Post to : [email protected] >>>>> Unsubscribe : >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>> >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Mailing list: >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>> Post to : [email protected] >>>> Unsubscribe : >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>> >>>> >>> >> >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
