> -----Original Message----- > From: Marcus Sorensen [mailto:shadow...@gmail.com] > Sent: Thursday, August 30, 2012 10:21 PM > To: Edison Su > Cc: cloudstack-dev@incubator.apache.org; Anthony Xu; Kelven Yang; > Vijayendra Bhamidipati > Subject: Re: VPC for KVM (was VM router spawning multiple public nics) > > I've made some good progress today, I'll probably have it all working > tomorrow, but I have a question. Looking at the API command > associateIpAddress and the functional spec, if I pass a vpcId it then > calls associateIPToVpc. Unlike its brother, associateIPToNetwork, I > don't clearly see where the ip is actually put on the router, I only > see it marking the ip allocated in the database. The functional spec > reads as though this IP should be on the VPC. Maybe I'm just not > getting that bolded statement 'All ip addresses get allocated to the > VPC', my first thought is that of course they would, any IP I allocate > gets put on the router... I guess I just need to play with it a bit > more.
After reading the functional spec, my guess is that: One VPC has multiple guest networks(one guest network is a tenant, e.g VLAN). On the guest network, there are a lot of VMs running on it. All of these VMs will use ip address range allocated from guest network(createnetwork api), usually they are using internal ip address. Regarding to these commands: 1. AssociateIPAddrCmd "in VPC setup as a result of associateIpAddress all ip addresses get allocated to VPC " means public ip address(when creating a zone, you'd specify public ip address range) are belong to a VPC, you can associate a public address to a guest network which attached to a VPC. In the router, it will add a public ip address on an ethX(which is created on public.network.device on kvm host), add a routing table(routing guest network gateway to that public ip). So after that, user VMs created on that guest network should be able to access public network through that public ip. 2. There are other commands(netwrok), which do some magic inside router vm, basically implement the features like, who can access this guest network, how to set up a portforwording rule for guest network etc. > > Not in front of the code at the moment, I just thought of a few more > quick questions that would help... What is the expected outcome of the > default rules? no access between private networks, but public access > for every network? By default, all incoming traffic to guest networks is blocked, but outgoing traffic should work. > > The functional spec mentions public and private gateways, does public > refer to giving someone say a /24 of public address space and then > creating a route on our router to their VPC router, i.e. public > networks instead of NATed ones? All of these gateways are referring to gateways inside router VM, not any physical router/gateway. The public refer to the network created on public.network.device on your kvm host. The private gateways referring to network created on guest.network.device on your kvm host. In a VPC setup, the router can have multiple guest network(a.k. multiple private gateways) and multiple public networks. You can call AssociateIPAddrCmd to bind a public ip to a guest network, then can implement portforwording etc. I am not VPC expert, my answer is not authoritative as it looks like:) Correct me if I am wrong. > > On Thu, Aug 30, 2012 at 11:54 AM, Marcus Sorensen <shadow...@gmail.com> > wrote: > > Yes, thanks for clarifying. > > > > On Thu, Aug 30, 2012 at 11:53 AM, Edison Su <edison...@citrix.com> > wrote: > >> Oh, I see, there are two kinds of commands: > >> One doesn't need to get information from libvirt, which just login > into router vm and program rules. For these commands, we can put them > into virtualRoutingResource. > >> Another does need to get infor from libvirt, such ipassoc, which > need to get nic infor from libvirt, and plug in vif if needed. For > these commands, need to be implemented in libvirtcomputingresource. > While if it needs to access/program router, then add put that code in > virtualroutingresource. > >> > >> Does it make sense? > >> > >>> -----Original Message----- > >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> Sent: Thursday, August 30, 2012 10:47 AM > >>> To: Edison Su > >>> Cc: cloudstack-dev@incubator.apache.org; Anthony Xu; Kelven Yang; > >>> Vijayendra Bhamidipati > >>> Subject: Re: VPC for KVM (was VM router spawning multiple public > nics) > >>> > >>> Ok, I think I get it. For example when I'm looking at > IpAssocCommand > >>> it calls _virtRouterResource.assignPublicIpAddress to actually run > the > >>> sh. So with SetupGuestNetworkCommand, I'll create that in > >>> LibvirtComputingResource, and within that I'll make a call to > >>> VirtualRoutingResource to do the work... is that right? > >>> > >>> On Thu, Aug 30, 2012 at 11:31 AM, Marcus Sorensen > <shadow...@gmail.com> > >>> wrote: > >>> > Just want to clarify: > >>> > > >>> > "We can add implementation of > >>> > > >>> > SetupGuestNetworkCommand/SetNetworkACLCommand/SetSourceNatCommand/Site2 > >>> SiteVpnCfgCommand > >>> > in VirtualRoutingResource." > >>> > > >>> > So the Xen implementation of SetupGuestNetworkCommand is in > >>> > CitrixResourceBase.java, and the VMware one is in > VmwareResource.java, > >>> > but you're saying I should put an implementation of these in > >>> > VirtualRoutingResource.java for KVM? Sorry, still trying to piece > >>> > together how everything relates. > >>> > > >>> > On Thu, Aug 30, 2012 at 10:42 AM, Edison Su > <edison...@citrix.com> > >>> wrote: > >>> >> Yes, it is. > >>> >> > >>> >>> -----Original Message----- > >>> >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> >>> Sent: Thursday, August 30, 2012 8:45 AM > >>> >>> To: Edison Su > >>> >>> Cc: cloudstack-dev@incubator.apache.org; Anthony Xu; Kelven > Yang; > >>> >>> Vijayendra Bhamidipati > >>> >>> Subject: VPC for KVM (was VM router spawning multiple public > nics) > >>> >>> > >>> >>> I notice there's also an 'IpAssocVpcCommand', I'm assuming that > >>> should > >>> >>> be added to the list? > >>> >>> > >>> >>> On Wed, Aug 29, 2012 at 6:37 PM, Edison Su > <edison...@citrix.com> > >>> wrote: > >>> >>> > You can find the VPC reference implementation from > >>> >>> CitrixResourceBase.java, which is the implementation for > Xenserver. > >>> >>> Just take a look at how the VPC related commands are > implemented. > >>> >>> > Take SetNetworkACLCommand as an example: > >>> >>> > The function execute(SetNetworkACLCommand cmd) in > >>> citrixResourceBase: > >>> >>> > 1. parse SetNetworkACLCommand > >>> >>> > 2. call scripts/vm/hypervisor/xenserver/vmops, function > >>> routerProxy > >>> >>> > 3. routerproxy will call scripts/network/domr/router_proxy.sh > >>> >>> > 4. router_proxy.sh will login into router vm, execute a shell > >>> script > >>> >>> inside router VM to program rules. > >>> >>> > > >>> >>> > In KVM, we can directly call router_proxy.sh or directly > login > >>> into > >>> >>> router vm, we just need to prepare the parameters for this > script. > >>> >>> > The reference code is in VirtualRoutingResource.java, all the > >>> network > >>> >>> related command(extended from NetworkElementCommand) will be > >>> redirected > >>> >>> to VirtualRoutingResource. > >>> >>> > We can add implementation of > >>> >>> > >>> > SetupGuestNetworkCommand/SetNetworkACLCommand/SetSourceNatCommand/Site2 > >>> >>> SiteVpnCfgCommand in VirtualRoutingResource. > >>> >>> > > >>> >>> > > >>> >>> >> -----Original Message----- > >>> >>> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> >>> >> Sent: Wednesday, August 29, 2012 4:34 PM > >>> >>> >> To: cloudstack-dev@incubator.apache.org > >>> >>> >> Cc: Edison Su; Anthony Xu; Kelven Yang; Vijayendra > Bhamidipati > >>> >>> >> Subject: Re: VM router spawning multiple public nics > >>> >>> >> > >>> >>> >> Great, thanks. I've already got a working implementation of > >>> >>> >> adding/removing nics from regular instances that I've been > >>> playing > >>> >>> >> with, so I'm getting vaguely familiar with the various data > >>> types > >>> >>> and > >>> >>> >> things surrounding the networking. I don't know if this is > quite > >>> >>> >> within my reach just yet but I'll see how far I get. > >>> >>> >> > >>> >>> >> On Wed, Aug 29, 2012 at 3:19 PM, Alena Prokharchyk > >>> >>> >> <alena.prokharc...@citrix.com> wrote: > >>> >>> >> > On 8/29/12 1:34 PM, "Edison Su" <edison...@citrix.com> > wrote: > >>> >>> >> > > >>> >>> >> >>Hi Anthony & Alena, > >>> >>> >> >> Could you help to provide information about VPC, how it > >>> works, > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > Here is the functional spec on the feature: > >>> >>> >> > > >>> >>> >> > wiki.cloudstack.org/display/RelOps/Inter- > >>> >>> VLAN+Routing+functional+spec > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > VpcVirtualNetworkApplianceManagerImpl is the manager > >>> responsible > >>> >>> for > >>> >>> >> VPC > >>> >>> >> > Virtual router operations (plug/unplug nics, etc) > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > > >>> >>> >> >> which commands needed to implemented on the hypervisor > side? > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > 1) PlugNicCommand/UnplugNicCommand - does Nic > hotplug/unplug > >>> >>> >> (currently > >>> >>> >> > works for VR vm only). In VPC called when add nic for > >>> Public/Guest > >>> >>> >> > networks. > >>> >>> >> > 2) SetupGuestNetworkCommand - sets up dhcp range, dns > >>> information, > >>> >>> >> > networkDomain information on the Nic to make it > >>> >>> >> > 3) SetNetworkACLCommand - creates network ACL on the > virtual > >>> >>> router > >>> >>> >> > 4) SetSourceNatCommand - used for setting source nat on > the > >>> Public > >>> >>> IP > >>> >>> >> on > >>> >>> >> > the VPC VR. > >>> >>> >> > 5) Site2SiteVpnCfgCommand - for setting up S2S VPN > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > Anthony/Kelven/Vijay did implementation for Xen/vmWare > >>> resources, > >>> >>> >> they can > >>> >>> >> > help you answering all hypervisor related questions. If > you > >>> need > >>> >>> more > >>> >>> >> > details on business logic + Vpc VR management, I can help > with > >>> >>> that. > >>> >>> >> > > >>> >>> >> > > >>> >>> >> > -Alena. > >>> >>> >> > > >>> >>> >> >> > >>> >>> >> >>> -----Original Message----- > >>> >>> >> >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> >>> >> >>> Sent: Wednesday, August 29, 2012 10:16 AM > >>> >>> >> >>> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> Subject: Re: VM router spawning multiple public nics > >>> >>> >> >>> > >>> >>> >> >>> I'd be willing to give it a shot if someone could point > me > >>> in > >>> >>> the > >>> >>> >> >>> right direction and be available to answer questions. > >>> >>> >> >>> > >>> >>> >> >>> On Wed, Aug 29, 2012 at 10:39 AM, Edison Su > >>> >>> <edison...@citrix.com> > >>> >>> >> >>> wrote: > >>> >>> >> >>> > Yah, KVM doesn't support VPC yet. Will you help to add > VPC > >>> >>> >> support on > >>> >>> >> >>> KVM?:) Just implement a few VPC related commands... > >>> >>> >> >>> > > >>> >>> >> >>> >> -----Original Message----- > >>> >>> >> >>> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> >>> >> >>> >> Sent: Wednesday, August 29, 2012 6:49 AM > >>> >>> >> >>> >> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> >> Subject: Re: VM router spawning multiple public nics > >>> >>> >> >>> >> > >>> >>> >> >>> >> I can confirm that the patch has fixed my particular > >>> issue. > >>> >>> >> >>> >> > >>> >>> >> >>> >> This is likely unrelated and I think it doesn't even > use > >>> the > >>> >>> >> same > >>> >>> >> >>> >> code, but I began to play with the VPC stuff a bit > and > >>> >>> noticed > >>> >>> >> that > >>> >>> >> >>> I > >>> >>> >> >>> >> don't get any interfaces except for link local. I'd > >>> probably > >>> >>> >> chalk > >>> >>> >> >>> >> that up to it not being ready for KVM, but I thought > it > >>> was > >>> >>> >> worth a > >>> >>> >> >>> >> mention. I'd be happy to try to help get it ready if > >>> someone > >>> >>> >> has > >>> >>> >> >>> time > >>> >>> >> >>> >> to nudge me in the right direction. > >>> >>> >> >>> >> > >>> >>> >> >>> >> On Tue, Aug 28, 2012 at 3:15 PM, Edison Su > >>> >>> >> <edison...@citrix.com> > >>> >>> >> >>> wrote: > >>> >>> >> >>> >> > > >>> >>> >> >>> >> > > >>> >>> >> >>> >> >> -----Original Message----- > >>> >>> >> >>> >> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >>> >>> >> >>> >> >> Sent: Tuesday, August 28, 2012 2:00 PM > >>> >>> >> >>> >> >> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> >> >> Subject: Re: VM router spawning multiple public > nics > >>> >>> >> >>> >> >> > >>> >>> >> >>> >> >> I thought about this solution myself, but below > this > >>> >>> portion > >>> >>> >> of > >>> >>> >> >>> >> code > >>> >>> >> >>> >> >> it looks like it uses the hash map to determine > which > >>> nic > >>> >>> >> number > >>> >>> >> >>> to > >>> >>> >> >>> >> >> add the IP to, so with multiple 'untagged' > networks it > >>> >>> would > >>> >>> >> have > >>> >>> >> >>> no > >>> >>> >> >>> >> >> way of knowing which nicnum in the router > corresponds > >>> with > >>> >>> >> the > >>> >>> >> >>> >> correct > >>> >>> >> >>> >> >> untagged vlan. > >>> >>> >> >>> >> >> > >>> >>> >> >>> >> >> nicNum = > >>> >>> >> vlanAllocatedToVM.get(ip.getVlanId()); > >>> >>> >> >>> >> >> networkUsage(routerIp, "addVif", > "eth" > >>> + > >>> >>> >> nicNum); > >>> >>> >> >>> >> >> result = > >>> >>> >> >>> >> >> > _virtRouterResource.assignPublicIpAddress(routerName, > >>> >>> >> >>> >> >> routerIp, ip.getPublicIp(), > >>> >>> >> ip.isAdd(), > >>> >>> >> >>> >> >> ip.isFirstIP(), > >>> >>> >> >>> >> >> ip.isSourceNat(), > >>> ip.getVlanId(), > >>> >>> >> >>> >> >> ip.getVlanGateway(), > >>> >>> >> >>> >> >> ip.getVlanNetmask(), > >>> >>> >> >>> ip.getVifMacAddress(), > >>> >>> >> >>> >> >> ip.getGuestIp(), nicNum); > >>> >>> >> >>> >> >> > >>> >>> >> >>> >> >> if ip.getVlanId() returns untagged (as it does on > >>> networks > >>> >>> >> with > >>> >>> >> >>> no > >>> >>> >> >>> >> >> vlan id), and we tried to put multiple untagged > keys > >>> in > >>> >>> >> >>> >> >> vlanAllocatedToVM (as with multiple untagged > networks), > >>> we > >>> >>> >> get > >>> >>> >> >>> the > >>> >>> >> >>> >> >> wrong nicNum, no? > >>> >>> >> >>> >> > > >>> >>> >> >>> >> > In the ipassoc case, if there are multiple untagged > >>> >>> networks, > >>> >>> >> all > >>> >>> >> >>> of > >>> >>> >> >>> >> them are use the same > >>> >>> >> >>> >> > Public bridge. Then multiple ip address will be > added > >>> on > >>> >>> eth2 > >>> >>> >> >>> inside > >>> >>> >> >>> >> router VM. > >>> >>> >> >>> >> > If it works physically, then it works. > >>> >>> >> >>> >> > > >>> >>> >> >>> >> >> > >>> >>> >> >>> >> >> On Tue, Aug 28, 2012 at 2:48 PM, Edison Su > >>> >>> >> <edison...@citrix.com> > >>> >>> >> >>> >> wrote: > >>> >>> >> >>> >> >> > > >>> >>> >> >>> >> >> > > >>> >>> >> >>> >> >> >> -----Original Message----- > >>> >>> >> >>> >> >> >> From: Marcus Sorensen > [mailto:shadow...@gmail.com] > >>> >>> >> >>> >> >> >> Sent: Tuesday, August 28, 2012 1:40 PM > >>> >>> >> >>> >> >> >> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> >> >> >> Subject: Re: VM router spawning multiple public > >>> nics > >>> >>> >> >>> >> >> >> > >>> >>> >> >>> >> >> >> Yes, that looks like it would work for me, > however > >>> >>> that's > >>> >>> >> not > >>> >>> >> >>> >> >> >> something that would ever make it into master, > >>> right? > >>> >>> >> >>> Essentially > >>> >>> >> >>> >> >> >> killing tagging for the public, private, and > guest > >>> >>> traffic > >>> >>> >> >>> labels? > >>> >>> >> >>> >> >> >> There's also still the issue of not being able > to > >>> >>> >> >>> differentiate > >>> >>> >> >>> >> >> >> between multiple untagged networks, if we > wanted to > >>> add > >>> >>> an > >>> >>> >> IP > >>> >>> >> >>> to > >>> >>> >> >>> >> a > >>> >>> >> >>> >> >> >> router it might not know which untagged > interface > >>> to > >>> >>> apply > >>> >>> >> it > >>> >>> >> >>> to. > >>> >>> >> >>> >> >> > > >>> >>> >> >>> >> >> > Physically, all the "untagged" network will be > >>> created > >>> >>> on > >>> >>> >> >>> >> >> public/guest/private bridge(the name we put in > >>> >>> >> >>> >> >> private/public/guest.bridge.name in > agent.properties"). > >>> >>> >> >>> >> >> > Because, there is no way to create a new > untagged > >>> bridge > >>> >>> by > >>> >>> >> >>> agent > >>> >>> >> >>> >> >> itself. Agent code only knows how to create a new > >>> >>> tagged(vlan) > >>> >>> >> >>> >> bridge. > >>> >>> >> >>> >> >> > So the fix should be pushed into master. > >>> >>> >> >>> >> >> > > >>> >>> >> >>> >> >> >> > >>> >>> >> >>> >> >> >> On Tue, Aug 28, 2012 at 2:32 PM, Edison Su > >>> >>> >> >>> <edison...@citrix.com> > >>> >>> >> >>> >> >> wrote: > >>> >>> >> >>> >> >> >> > > >>> >>> >> >>> >> >> >> > > >>> >>> >> >>> >> >> >> >> -----Original Message----- > >>> >>> >> >>> >> >> >> >> From: Marcus Sorensen > >>> [mailto:shadow...@gmail.com] > >>> >>> >> >>> >> >> >> >> Sent: Tuesday, August 28, 2012 12:23 PM > >>> >>> >> >>> >> >> >> >> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> >> >> >> >> Subject: Re: VM router spawning multiple > public > >>> nics > >>> >>> >> >>> >> >> >> >> > >>> >>> >> >>> >> >> >> >> Thanks for pointing me in the right > direction. > >>> I've > >>> >>> >> >>> reviewed > >>> >>> >> >>> >> this > >>> >>> >> >>> >> >> >> code > >>> >>> >> >>> >> >> >> >> in a bit more detail, and it seems like it's > >>> >>> >> accomplishing > >>> >>> >> >>> the > >>> >>> >> >>> >> >> >> >> following: > >>> >>> >> >>> >> >> >> >> > >>> >>> >> >>> >> >> >> >> 1. get all network interfaces currently > >>> connected to > >>> >>> >> the > >>> >>> >> >>> >> running > >>> >>> >> >>> >> >> VM > >>> >>> >> >>> >> >> >> >> (a.k.a vnet devices via libvirt) > >>> >>> >> >>> >> >> >> >> 2. find out which vlans these network > interfaces > >>> are > >>> >>> >> >>> bridged > >>> >>> >> >>> >> to, > >>> >>> >> >>> >> >> >> store > >>> >>> >> >>> >> >> >> >> this in a hash map of vlan ids and nics > >>> >>> >> >>> >> >> >> >> 3. get all ip addresses to be added to the > VM > >>> >>> >> >>> >> >> >> >> 4. for each ip, get the configured vlan id > for > >>> the > >>> >>> ip, > >>> >>> >> >>> compare > >>> >>> >> >>> >> it > >>> >>> >> >>> >> >> to > >>> >>> >> >>> >> >> >> >> the hash map of existing vlan ids and nics > >>> >>> >> >>> >> >> >> >> 5. if the required vlan id is not found in > the > >>> hash > >>> >>> map, > >>> >>> >> >>> >> create a > >>> >>> >> >>> >> >> >> new > >>> >>> >> >>> >> >> >> >> nic > >>> >>> >> >>> >> >> >> >> 6. assign the ip to the nic identified by > the > >>> vlan > >>> >>> id > >>> >>> >> key > >>> >>> >> >>> in > >>> >>> >> >>> >> the > >>> >>> >> >>> >> >> >> hash > >>> >>> >> >>> >> >> >> >> map > >>> >>> >> >>> >> >> >> >> > >>> >>> >> >>> >> >> >> >> In this case, we're getting a vlan id > returned > >>> in > >>> >>> step > >>> >>> >> 2 > >>> >>> >> >>> for a > >>> >>> >> >>> >> >> >> bridged > >>> >>> >> >>> >> >> >> >> nic whose network is defined as untagged in > the > >>> >>> >> cloudstack > >>> >>> >> >>> db, > >>> >>> >> >>> >> >> >> >> therefore in step 5 we never match as > already > >>> having > >>> >>> a > >>> >>> >> nic > >>> >>> >> >>> for > >>> >>> >> >>> >> >> >> >> 'untagged'. I wrote a big long response > >>> discussing > >>> >>> this > >>> >>> >> >>> issue, > >>> >>> >> >>> >> >> but > >>> >>> >> >>> >> >> >> as > >>> >>> >> >>> >> >> >> >> I began to dig further I realized that aside > >>> from my > >>> >>> >> >>> >> particular > >>> >>> >> >>> >> >> case, > >>> >>> >> >>> >> >> >> >> untagged vlans in general are just broken > (for > >>> >>> example > >>> >>> >> they > >>> >>> >> >>> >> can't > >>> >>> >> >>> >> >> be > >>> >>> >> >>> >> >> >> >> dealt with uniquely in the current > >>> IpAssocCommand > >>> >>> code, > >>> >>> >> >>> given > >>> >>> >> >>> >> the > >>> >>> >> >>> >> >> >> hash > >>> >>> >> >>> >> >> >> >> map) and it would require more effort than I > >>> have > >>> >>> time > >>> >>> >> for > >>> >>> >> >>> now > >>> >>> >> >>> >> to > >>> >>> >> >>> >> >> >> make > >>> >>> >> >>> >> >> >> >> things work. If the code were already in > place > >>> to > >>> >>> >> >>> >> differentiate > >>> >>> >> >>> >> >> >> >> between multiple untagged nics I think that > >>> fixing > >>> >>> my > >>> >>> >> >>> problem > >>> >>> >> >>> >> >> would > >>> >>> >> >>> >> >> >> be > >>> >>> >> >>> >> >> >> >> trivial, but since its not, I'll just find > an > >>> >>> >> alternative > >>> >>> >> >>> >> >> solution. > >>> >>> >> >>> >> >> >> >> > >>> >>> >> >>> >> >> >> > > >>> >>> >> >>> >> >> >> > The untagged network usually means "untagged", > no > >>> >>> vlan > >>> >>> >> on > >>> >>> >> >>> the > >>> >>> >> >>> >> >> >> bridge... > >>> >>> >> >>> >> >> >> > In your case, the untagged network actually > has > >>> >>> >> vlan(tagged) > >>> >>> >> >>> on > >>> >>> >> >>> >> >> the > >>> >>> >> >>> >> >> >> bridge, thus getting things confused. > >>> >>> >> >>> >> >> >> > Will this patch(http://pastebin.com/HJXzZwKp) > >>> work > >>> >>> for > >>> >>> >> you? > >>> >>> >> >>> >> >> >> > > >>> >>> >> >>> >> >> >> >> On Mon, Aug 27, 2012 at 10:46 PM, Marcus > >>> Sorensen > >>> >>> >> >>> >> >> >> <shadow...@gmail.com> > >>> >>> >> >>> >> >> >> >> wrote: > >>> >>> >> >>> >> >> >> >> > ... > >>> >>> >> >>> >> >> >> >> > Integer nicPos = 0; > >>> >>> >> >>> >> >> >> >> > for (InterfaceDef nic : nics) > { > >>> >>> >> >>> >> >> >> >> > if > >>> >>> >> >>> >> >> >> >> > >>> >>> (nic.getBrName().equalsIgnoreCase(_linkLocalBridgeName)) > >>> >>> >> { > >>> >>> >> >>> >> >> >> >> > > >>> >>> vlanAllocatedToVM.put("LinkLocal", > >>> >>> >> >>> >> nicPos); > >>> >>> >> >>> >> >> >> >> > } else { > >>> >>> >> >>> >> >> >> >> > String vlanId = > >>> >>> >> >>> >> >> >> >> getVlanIdFromBridge(nic.getBrName()); > >>> >>> >> >>> >> >> >> >> > if (vlanId != null) { > >>> >>> >> >>> >> >> >> >> > > >>> >>> vlanAllocatedToVM.put(vlanId, > >>> >>> >> >>> >> nicPos); > >>> >>> >> >>> >> >> >> >> > } else { > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> vlanAllocatedToVM.put(Vlan.UNTAGGED, > >>> >>> >> >>> >> >> >> nicPos); > >>> >>> >> >>> >> >> >> >> > } > >>> >>> >> >>> >> >> >> >> > } > >>> >>> >> >>> >> >> >> >> > nicPos++; > >>> >>> >> >>> >> >> >> >> > } > >>> >>> >> >>> >> >> >> >> > IpAddressTO[] ips = > >>> >>> cmd.getIpAddresses(); > >>> >>> >> >>> >> >> >> >> > int i = 0; > >>> >>> >> >>> >> >> >> >> > String result = null; > >>> >>> >> >>> >> >> >> >> > int nicNum = 0; > >>> >>> >> >>> >> >> >> >> > for (IpAddressTO ip : ips) { > >>> >>> >> >>> >> >> >> >> > if > >>> >>> >> >>> >> >> (!vlanAllocatedToVM.containsKey(ip.getVlanId())) > >>> >>> >> >>> >> >> >> { > >>> >>> >> >>> >> >> >> >> > /* plug a vif into > router > >>> */ > >>> >>> >> >>> >> >> >> >> > VifHotPlug(conn, > >>> routerName, > >>> >>> >> >>> >> ip.getVlanId(), > >>> >>> >> >>> >> >> >> >> > > >>> ip.getVifMacAddress()); > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> vlanAllocatedToVM.put(ip.getVlanId(), > >>> >>> >> >>> >> >> >> nicPos++); > >>> >>> >> >>> >> >> >> >> > } > >>> >>> >> >>> >> >> >> >> > ... > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > Looks like the getVlanIdFromBridge might > be a > >>> bit > >>> >>> >> >>> misleading. > >>> >>> >> >>> >> I > >>> >>> >> >>> >> >> am > >>> >>> >> >>> >> >> >> >> > running my guest public traffic on a > >>> 'cloudbr470', > >>> >>> >> which > >>> >>> >> >>> is > >>> >>> >> >>> >> a > >>> >>> >> >>> >> >> >> bridge > >>> >>> >> >>> >> >> >> >> > to eth2.470, yet I configured this network > as > >>> >>> >> 'untagged' > >>> >>> >> >>> >> >> because I > >>> >>> >> >>> >> >> >> >> > have a vlan 470 available on eth3 for > >>> cloudstack > >>> >>> to > >>> >>> >> >>> >> autoassign > >>> >>> >> >>> >> >> >> (eth3 > >>> >>> >> >>> >> >> >> >> > is where all of my stuff will be > autoassigned). > >>> So > >>> >>> >> I'm > >>> >>> >> >>> not > >>> >>> >> >>> >> 100% > >>> >>> >> >>> >> >> >> sure > >>> >>> >> >>> >> >> >> >> > yet what's going on here but it seems as > >>> though > >>> >>> the > >>> >>> >> above > >>> >>> >> >>> is > >>> >>> >> >>> >> >> not > >>> >>> >> >>> >> >> >> >> > setting any 'Vlan.UNTAGGED', since it > finds a > >>> vlan > >>> >>> >> number > >>> >>> >> >>> >> for > >>> >>> >> >>> >> >> >> >> > eth2.470, but when it enumerates the IPs > for > >>> the > >>> >>> >> router, > >>> >>> >> >>> it > >>> >>> >> >>> >> >> then > >>> >>> >> >>> >> >> >> runs > >>> >>> >> >>> >> >> >> >> > ip.getVlanId() and doesn't find a nic for > the > >>> >>> >> untagged IP > >>> >>> >> >>> >> and > >>> >>> >> >>> >> >> >> creates > >>> >>> >> >>> >> >> >> >> > one. > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > I realize this is perhaps an uncommon case, > >>> but a > >>> >>> bug > >>> >>> >> >>> >> >> nonetheless. > >>> >>> >> >>> >> >> >> >> > I'll play with the code a bit and see if I > can > >>> >>> come > >>> >>> >> up > >>> >>> >> >>> with > >>> >>> >> >>> >> a > >>> >>> >> >>> >> >> >> >> > solution. I'm thinking I can look at the > nic's > >>> >>> >> broadcast > >>> >>> >> >>> URI > >>> >>> >> >>> >> >> and > >>> >>> >> >>> >> >> >> see > >>> >>> >> >>> >> >> >> >> > if it's supposed to be untagged, then add > to > >>> >>> >> >>> >> vlanAllocatedToVM > >>> >>> >> >>> >> >> >> >> > appropriately, off the top of my head > >>> something > >>> >>> like: > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > String vlanId = > >>> >>> >> >>> >> >> >> >> getVlanIdFromBridge(nic.getBrName()); > >>> >>> >> >>> >> >> >> >> > if (vlanId != null && > >>> >>> >> >>> >> >> >> >> > >>> >>> > !nic.getBroadcastUri().toString().contains("untagged") > >>> >>> >> { > >>> >>> >> >>> >> >> >> >> > > >>> >>> vlanAllocatedToVM.put(vlanId, > >>> >>> >> >>> >> nicPos); > >>> >>> >> >>> >> >> >> >> > } else { > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> vlanAllocatedToVM.put(Vlan.UNTAGGED, > >>> >>> >> >>> >> >> >> nicPos); > >>> >>> >> >>> >> >> >> >> > } > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > > >>> >>> >> >>> >> >> >> >> > On Mon, Aug 27, 2012 at 6:42 PM, Edison Su > >>> >>> >> >>> >> >> <edison...@citrix.com> > >>> >>> >> >>> >> >> >> >> wrote: > >>> >>> >> >>> >> >> >> >> >> Possible bug in in kvm code: > >>> >>> >> LibvirtComputingResource- > >>> >>> >> >>> >> >> >> >> >execute(IpAssocCommand cmd)-> VifHotPlug, > which > >>> is > >>> >>> >> only > >>> >>> >> >>> place > >>> >>> >> >>> >> >> >> adding > >>> >>> >> >>> >> >> >> >> nic into router vm. > >>> >>> >> >>> >> >> >> >> >> Turn on agent log, then take a look what > >>> happened. > >>> >>> >> >>> >> >> >> >> >> > >>> >>> >> >>> >> >> >> >> >>> -----Original Message----- > >>> >>> >> >>> >> >> >> >> >>> From: Marcus Sorensen > >>> >>> [mailto:shadow...@gmail.com] > >>> >>> >> >>> >> >> >> >> >>> Sent: Monday, August 27, 2012 5:10 PM > >>> >>> >> >>> >> >> >> >> >>> To: cloudstack-dev@incubator.apache.org > >>> >>> >> >>> >> >> >> >> >>> Subject: VM router spawning multiple > public > >>> nics > >>> >>> >> >>> >> >> >> >> >>> > >>> >>> >> >>> >> >> >> >> >>> I've got two zones running the same > build of > >>> >>> >> cloudstack > >>> >>> >> >>> (a > >>> >>> >> >>> >> >> >> recent > >>> >>> >> >>> >> >> >> >> copy > >>> >>> >> >>> >> >> >> >> >>> of master). One of them creates routers > that > >>> >>> turn > >>> >>> >> into > >>> >>> >> >>> >> ugly > >>> >>> >> >>> >> >> >> >> >>> multi-headed beasts, and by that I mean > that > >>> any > >>> >>> >> time I > >>> >>> >> >>> >> >> create a > >>> >>> >> >>> >> >> >> >> port > >>> >>> >> >>> >> >> >> >> >>> forwarding or iptables rule for that > router > >>> I > >>> >>> get a > >>> >>> >> new > >>> >>> >> >>> >> >> public > >>> >>> >> >>> >> >> >> NIC > >>> >>> >> >>> >> >> >> >> >>> with an identical IP address, I have an > >>> instance > >>> >>> >> with a > >>> >>> >> >>> >> few > >>> >>> >> >>> >> >> tens > >>> >>> >> >>> >> >> >> of > >>> >>> >> >>> >> >> >> >> >>> NICs. My guess is that some script > isn't > >>> >>> detecting > >>> >>> >> >>> that > >>> >>> >> >>> >> >> there's > >>> >>> >> >>> >> >> >> >> >>> already a NIC with the public IP on it. > It > >>> >>> looks > >>> >>> >> fine > >>> >>> >> >>> in > >>> >>> >> >>> >> the > >>> >>> >> >>> >> >> >> >> >>> database, there is only one public NIC > >>> defined > >>> >>> in > >>> >>> >> the > >>> >>> >> >>> nics > >>> >>> >> >>> >> >> table. > >>> >>> >> >>> >> >> >> >> >>> I'll troubleshoot it tomorrow, but if > anyone > >>> >>> knows > >>> >>> >> >>> where I > >>> >>> >> >>> >> >> >> should > >>> >>> >> >>> >> >> >> >> >>> begin the headstart would be appreciated. > >>> >>> >> >>> >> >> >> >> >>> > >>> >>> >> >>> >> >> >> >> >>> Thanks > >>> >>> >> >> > >>> >>> >> > > >>> >>> >> >
