> -----Original Message----- > From: Marcus Sorensen [mailto:shadow...@gmail.com] > Sent: Tuesday, August 28, 2012 1:40 PM > To: cloudstack-dev@incubator.apache.org > Subject: Re: VM router spawning multiple public nics > > Yes, that looks like it would work for me, however that's not > something that would ever make it into master, right? Essentially > killing tagging for the public, private, and guest traffic labels? > There's also still the issue of not being able to differentiate > between multiple untagged networks, if we wanted to add an IP to a > router it might not know which untagged interface to apply it to.
Physically, all the "untagged" network will be created on public/guest/private bridge(the name we put in private/public/guest.bridge.name in agent.properties"). Because, there is no way to create a new untagged bridge by agent itself. Agent code only knows how to create a new tagged(vlan) bridge. So the fix should be pushed into master. > > On Tue, Aug 28, 2012 at 2:32 PM, Edison Su <edison...@citrix.com> wrote: > > > > > >> -----Original Message----- > >> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >> Sent: Tuesday, August 28, 2012 12:23 PM > >> To: cloudstack-dev@incubator.apache.org > >> Subject: Re: VM router spawning multiple public nics > >> > >> Thanks for pointing me in the right direction. I've reviewed this > code > >> in a bit more detail, and it seems like it's accomplishing the > >> following: > >> > >> 1. get all network interfaces currently connected to the running VM > >> (a.k.a vnet devices via libvirt) > >> 2. find out which vlans these network interfaces are bridged to, > store > >> this in a hash map of vlan ids and nics > >> 3. get all ip addresses to be added to the VM > >> 4. for each ip, get the configured vlan id for the ip, compare it to > >> the hash map of existing vlan ids and nics > >> 5. if the required vlan id is not found in the hash map, create a > new > >> nic > >> 6. assign the ip to the nic identified by the vlan id key in the > hash > >> map > >> > >> In this case, we're getting a vlan id returned in step 2 for a > bridged > >> nic whose network is defined as untagged in the cloudstack db, > >> therefore in step 5 we never match as already having a nic for > >> 'untagged'. I wrote a big long response discussing this issue, but > as > >> I began to dig further I realized that aside from my particular case, > >> untagged vlans in general are just broken (for example they can't be > >> dealt with uniquely in the current IpAssocCommand code, given the > hash > >> map) and it would require more effort than I have time for now to > make > >> things work. If the code were already in place to differentiate > >> between multiple untagged nics I think that fixing my problem would > be > >> trivial, but since its not, I'll just find an alternative solution. > >> > > > > The untagged network usually means "untagged", no vlan on the > bridge... > > In your case, the untagged network actually has vlan(tagged) on the > bridge, thus getting things confused. > > Will this patch(http://pastebin.com/HJXzZwKp) work for you? > > > >> On Mon, Aug 27, 2012 at 10:46 PM, Marcus Sorensen > <shadow...@gmail.com> > >> wrote: > >> > ... > >> > Integer nicPos = 0; > >> > for (InterfaceDef nic : nics) { > >> > if > >> (nic.getBrName().equalsIgnoreCase(_linkLocalBridgeName)) { > >> > vlanAllocatedToVM.put("LinkLocal", nicPos); > >> > } else { > >> > String vlanId = > >> getVlanIdFromBridge(nic.getBrName()); > >> > if (vlanId != null) { > >> > vlanAllocatedToVM.put(vlanId, nicPos); > >> > } else { > >> > vlanAllocatedToVM.put(Vlan.UNTAGGED, > nicPos); > >> > } > >> > } > >> > nicPos++; > >> > } > >> > IpAddressTO[] ips = cmd.getIpAddresses(); > >> > int i = 0; > >> > String result = null; > >> > int nicNum = 0; > >> > for (IpAddressTO ip : ips) { > >> > if (!vlanAllocatedToVM.containsKey(ip.getVlanId())) > { > >> > /* plug a vif into router */ > >> > VifHotPlug(conn, routerName, ip.getVlanId(), > >> > ip.getVifMacAddress()); > >> > vlanAllocatedToVM.put(ip.getVlanId(), > nicPos++); > >> > } > >> > ... > >> > > >> > Looks like the getVlanIdFromBridge might be a bit misleading. I am > >> > running my guest public traffic on a 'cloudbr470', which is a > bridge > >> > to eth2.470, yet I configured this network as 'untagged' because I > >> > have a vlan 470 available on eth3 for cloudstack to autoassign > (eth3 > >> > is where all of my stuff will be autoassigned). So I'm not 100% > sure > >> > yet what's going on here but it seems as though the above is not > >> > setting any 'Vlan.UNTAGGED', since it finds a vlan number for > >> > eth2.470, but when it enumerates the IPs for the router, it then > runs > >> > ip.getVlanId() and doesn't find a nic for the untagged IP and > creates > >> > one. > >> > > >> > > >> > I realize this is perhaps an uncommon case, but a bug nonetheless. > >> > I'll play with the code a bit and see if I can come up with a > >> > solution. I'm thinking I can look at the nic's broadcast URI and > see > >> > if it's supposed to be untagged, then add to vlanAllocatedToVM > >> > appropriately, off the top of my head something like: > >> > > >> > String vlanId = > >> getVlanIdFromBridge(nic.getBrName()); > >> > if (vlanId != null && > >> > !nic.getBroadcastUri().toString().contains("untagged") { > >> > vlanAllocatedToVM.put(vlanId, nicPos); > >> > } else { > >> > vlanAllocatedToVM.put(Vlan.UNTAGGED, > nicPos); > >> > } > >> > > >> > > >> > > >> > On Mon, Aug 27, 2012 at 6:42 PM, Edison Su <edison...@citrix.com> > >> wrote: > >> >> Possible bug in in kvm code: LibvirtComputingResource- > >> >execute(IpAssocCommand cmd)-> VifHotPlug, which is only place > adding > >> nic into router vm. > >> >> Turn on agent log, then take a look what happened. > >> >> > >> >>> -----Original Message----- > >> >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >> >>> Sent: Monday, August 27, 2012 5:10 PM > >> >>> To: cloudstack-dev@incubator.apache.org > >> >>> Subject: VM router spawning multiple public nics > >> >>> > >> >>> I've got two zones running the same build of cloudstack (a > recent > >> copy > >> >>> of master). One of them creates routers that turn into ugly > >> >>> multi-headed beasts, and by that I mean that any time I create a > >> port > >> >>> forwarding or iptables rule for that router I get a new public > NIC > >> >>> with an identical IP address, I have an instance with a few tens > of > >> >>> NICs. My guess is that some script isn't detecting that there's > >> >>> already a NIC with the public IP on it. It looks fine in the > >> >>> database, there is only one public NIC defined in the nics table. > >> >>> I'll troubleshoot it tomorrow, but if anyone knows where I > should > >> >>> begin the headstart would be appreciated. > >> >>> > >> >>> Thanks
