We saw this issue in Grizzly as well. I investigated the Quantum logs and I did not find anything bad. The VM actually does get two interfaces in this case, so it seemed like some race condition on the nova side:
<interface type="bridge"> <mac address="fa:16:3e:a8:fe:96"/> <model type="virtio"/> <driver name="qemu"/> <source bridge="qbr7aac0341-19"/> <target dev="tap7aac0341-19"/> <filterref filter="nova-instance-instance-00000004-fa163ea8fe96"> <parameter name="IP" value="10.194.193.4"/> <parameter name="DHCPSERVER" value="10.194.193.2"/> <parameter name="PROJNET" value="10.194.193.0"/> <parameter name="PROJMASK" value="255.255.255.0"/> </filterref> </interface> <interface type="bridge"> <mac address="fa:16:3e:9a:b9:33"/> <model type="virtio"/> <driver name="qemu"/> <source bridge="qbr7ddce382-c7"/> <target dev="tap7ddce382-c7"/> <filterref filter="nova-instance-instance-00000004-fa163e9ab933"> <parameter name="IP" value="10.194.193.5"/> <parameter name="DHCPSERVER" value="10.194.193.2"/> <parameter name="PROJNET" value="10.194.193.0"/> <parameter name="PROJMASK" value="255.255.255.0"/> </filterref> </interface> Thanks, ~Sumit. On Tue, Mar 26, 2013 at 10:24 AM, Salvatore Orlando <sorla...@nicira.com> wrote: > The reschedule process is apparently safe (at least from my experience). > I'm not sure how much the sequentiality of the IPs might be a hint of > a different problem, as in the lp answer I see a case where the > duplicated addresses are not sequential. > Also, the script that is launching these VMs might spawn requests in > parallel, and so even without reschedule event there should not be any > guarantee about sequentiality of the resulting IPs. > > It's interesting to understand whether the total number of port is the > same as the total number of VM, ie: is there any VM without any port? > > Salvatore > > On 26 March 2013 17:45, Dan Wendlandt <d...@nicira.com> wrote: >> >> >> On Tue, Mar 26, 2013 at 9:36 AM, Gary Kotton <gkot...@redhat.com> wrote: >>> >>> Hi, >>> I have seen something like this with stable folsom. We have yet to be able >>> to reproduce it. In our setup we saw that there were timeouts with the >>> quantum service. In addition to this we had 2 compute nodes. My gut feeling >>> was that one of the nodes has a failer and the scheduler selects another >>> node. The second node will allocate another port. >>> In allocate instance >>> https://github.com/openstack/nova/blob/master/nova/network/quantumv2/api.py#L129. >>> I think tht we should check that a port already exists on the requested >>> network. >>> I am away with my family so will not have a chance to look at this till I >>> get back. >> >> >> Yeah, that was my initial thought as well, though in the case that was >> reported in the but, the IPs are sequential is what suggested to me that it >> may not be a "reschedule", since the user is spinning up many VMs at once, >> so sequential IPs would seem unlikely if there was a lot of lag time between >> the creation of the first port and second port. >> >> It may make sense to have a check that deletes any previous ports with the >> same device-id before allocating new ones (I probably wouldn't just re-use >> it, as we don't quite know what state it may be in). >> >> dan >> >> >>> >>> Sorry >>> Gary >>> >>> >>> >>> On 03/26/2013 05:58 PM, Dan Wendlandt wrote: >>> >>> This is interesting. I'll be in customer meetings and flying for the next >>> few hours, so I thought I'd send it out in case anyone else has time to >>> investigate first. >>> >>> https://bugs.launchpad.net/quantum/+bug/1160442 >>> >>> for details, see the associated question: >>> https://answers.launchpad.net/quantum/+question/225158 >>> >>> Also, we discussed this a while back, but we should double-check that >>> there is no risk with a malicious user changing the device_id of its port to >>> match another VM of another tenant, causing confusion by making that tenant >>> suddenly see multiple floating IPs for its VM. I think at the time we >>> decided that all queries from nova to quantum were properly scoped by >>> tenant, but it is worth make sure. >>> >>> Dan >>> >>> -- >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> Dan Wendlandt >>> Nicira, Inc: www.nicira.com >>> twitter: danwendlandt >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> >>> >>> >> >> >> >> -- >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> Dan Wendlandt >> Nicira, Inc: www.nicira.com >> twitter: danwendlandt >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> -- >> Mailing list: https://launchpad.net/~quantum-core >> Post to : quantum-core@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~quantum-core >> More help : https://help.launchpad.net/ListHelp >> > > -- > Mailing list: https://launchpad.net/~quantum-core > Post to : quantum-core@lists.launchpad.net > Unsubscribe : https://launchpad.net/~quantum-core > More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~quantum-core Post to : quantum-core@lists.launchpad.net Unsubscribe : https://launchpad.net/~quantum-core More help : https://help.launchpad.net/ListHelp
