I'm running into several little problems, like the vpc_guestnw.sh having an invalid logger_it function (changed to logger -t cloud), and apache being setup to listen on a non-existent 10.1.1.1:80 and 443 (deleted default-000 and default-ssl). Should I make a patch for vpc_guestnw.sh, or do I just have an out of date systemvm image or something?
On Thu, Aug 30, 2012 at 11:20 PM, Marcus Sorensen <shadow...@gmail.com> wrote: > I've made some good progress today, I'll probably have it all working > tomorrow, but I have a question. Looking at the API command > associateIpAddress and the functional spec, if I pass a vpcId it then > calls associateIPToVpc. Unlike its brother, associateIPToNetwork, I > don't clearly see where the ip is actually put on the router, I only > see it marking the ip allocated in the database. The functional spec > reads as though this IP should be on the VPC. Maybe I'm just not > getting that bolded statement 'All ip addresses get allocated to the > VPC', my first thought is that of course they would, any IP I allocate > gets put on the router... I guess I just need to play with it a bit > more. > > Not in front of the code at the moment, I just thought of a few more > quick questions that would help... What is the expected outcome of the > default rules? no access between private networks, but public access > for every network? > > The functional spec mentions public and private gateways, does public > refer to giving someone say a /24 of public address space and then > creating a route on our router to their VPC router, i.e. public > networks instead of NATed ones? > > On Thu, Aug 30, 2012 at 11:54 AM, Marcus Sorensen <shadow...@gmail.com> wrote: >> Yes, thanks for clarifying. >> >> On Thu, Aug 30, 2012 at 11:53 AM, Edison Su <edison...@citrix.com> wrote: >>> Oh, I see, there are two kinds of commands: >>> One doesn't need to get information from libvirt, which just login into >>> router vm and program rules. For these commands, we can put them into >>> virtualRoutingResource. >>> Another does need to get infor from libvirt, such ipassoc, which need to >>> get nic infor from libvirt, and plug in vif if needed. For these commands, >>> need to be implemented in libvirtcomputingresource. While if it needs to >>> access/program router, then add put that code in virtualroutingresource. >>> >>> Does it make sense? >>> >>>> -----Original Message----- >>>> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> Sent: Thursday, August 30, 2012 10:47 AM >>>> To: Edison Su >>>> Cc: cloudstack-dev@incubator.apache.org; Anthony Xu; Kelven Yang; >>>> Vijayendra Bhamidipati >>>> Subject: Re: VPC for KVM (was VM router spawning multiple public nics) >>>> >>>> Ok, I think I get it. For example when I'm looking at IpAssocCommand >>>> it calls _virtRouterResource.assignPublicIpAddress to actually run the >>>> sh. So with SetupGuestNetworkCommand, I'll create that in >>>> LibvirtComputingResource, and within that I'll make a call to >>>> VirtualRoutingResource to do the work... is that right? >>>> >>>> On Thu, Aug 30, 2012 at 11:31 AM, Marcus Sorensen <shadow...@gmail.com> >>>> wrote: >>>> > Just want to clarify: >>>> > >>>> > "We can add implementation of >>>> > >>>> SetupGuestNetworkCommand/SetNetworkACLCommand/SetSourceNatCommand/Site2 >>>> SiteVpnCfgCommand >>>> > in VirtualRoutingResource." >>>> > >>>> > So the Xen implementation of SetupGuestNetworkCommand is in >>>> > CitrixResourceBase.java, and the VMware one is in VmwareResource.java, >>>> > but you're saying I should put an implementation of these in >>>> > VirtualRoutingResource.java for KVM? Sorry, still trying to piece >>>> > together how everything relates. >>>> > >>>> > On Thu, Aug 30, 2012 at 10:42 AM, Edison Su <edison...@citrix.com> >>>> wrote: >>>> >> Yes, it is. >>>> >> >>>> >>> -----Original Message----- >>>> >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> Sent: Thursday, August 30, 2012 8:45 AM >>>> >>> To: Edison Su >>>> >>> Cc: cloudstack-dev@incubator.apache.org; Anthony Xu; Kelven Yang; >>>> >>> Vijayendra Bhamidipati >>>> >>> Subject: VPC for KVM (was VM router spawning multiple public nics) >>>> >>> >>>> >>> I notice there's also an 'IpAssocVpcCommand', I'm assuming that >>>> should >>>> >>> be added to the list? >>>> >>> >>>> >>> On Wed, Aug 29, 2012 at 6:37 PM, Edison Su <edison...@citrix.com> >>>> wrote: >>>> >>> > You can find the VPC reference implementation from >>>> >>> CitrixResourceBase.java, which is the implementation for Xenserver. >>>> >>> Just take a look at how the VPC related commands are implemented. >>>> >>> > Take SetNetworkACLCommand as an example: >>>> >>> > The function execute(SetNetworkACLCommand cmd) in >>>> citrixResourceBase: >>>> >>> > 1. parse SetNetworkACLCommand >>>> >>> > 2. call scripts/vm/hypervisor/xenserver/vmops, function >>>> routerProxy >>>> >>> > 3. routerproxy will call scripts/network/domr/router_proxy.sh >>>> >>> > 4. router_proxy.sh will login into router vm, execute a shell >>>> script >>>> >>> inside router VM to program rules. >>>> >>> > >>>> >>> > In KVM, we can directly call router_proxy.sh or directly login >>>> into >>>> >>> router vm, we just need to prepare the parameters for this script. >>>> >>> > The reference code is in VirtualRoutingResource.java, all the >>>> network >>>> >>> related command(extended from NetworkElementCommand) will be >>>> redirected >>>> >>> to VirtualRoutingResource. >>>> >>> > We can add implementation of >>>> >>> >>>> SetupGuestNetworkCommand/SetNetworkACLCommand/SetSourceNatCommand/Site2 >>>> >>> SiteVpnCfgCommand in VirtualRoutingResource. >>>> >>> > >>>> >>> > >>>> >>> >> -----Original Message----- >>>> >>> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> >> Sent: Wednesday, August 29, 2012 4:34 PM >>>> >>> >> To: cloudstack-dev@incubator.apache.org >>>> >>> >> Cc: Edison Su; Anthony Xu; Kelven Yang; Vijayendra Bhamidipati >>>> >>> >> Subject: Re: VM router spawning multiple public nics >>>> >>> >> >>>> >>> >> Great, thanks. I've already got a working implementation of >>>> >>> >> adding/removing nics from regular instances that I've been >>>> playing >>>> >>> >> with, so I'm getting vaguely familiar with the various data >>>> types >>>> >>> and >>>> >>> >> things surrounding the networking. I don't know if this is quite >>>> >>> >> within my reach just yet but I'll see how far I get. >>>> >>> >> >>>> >>> >> On Wed, Aug 29, 2012 at 3:19 PM, Alena Prokharchyk >>>> >>> >> <alena.prokharc...@citrix.com> wrote: >>>> >>> >> > On 8/29/12 1:34 PM, "Edison Su" <edison...@citrix.com> wrote: >>>> >>> >> > >>>> >>> >> >>Hi Anthony & Alena, >>>> >>> >> >> Could you help to provide information about VPC, how it >>>> works, >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > Here is the functional spec on the feature: >>>> >>> >> > >>>> >>> >> > wiki.cloudstack.org/display/RelOps/Inter- >>>> >>> VLAN+Routing+functional+spec >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > VpcVirtualNetworkApplianceManagerImpl is the manager >>>> responsible >>>> >>> for >>>> >>> >> VPC >>>> >>> >> > Virtual router operations (plug/unplug nics, etc) >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> >> which commands needed to implemented on the hypervisor side? >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > 1) PlugNicCommand/UnplugNicCommand - does Nic hotplug/unplug >>>> >>> >> (currently >>>> >>> >> > works for VR vm only). In VPC called when add nic for >>>> Public/Guest >>>> >>> >> > networks. >>>> >>> >> > 2) SetupGuestNetworkCommand - sets up dhcp range, dns >>>> information, >>>> >>> >> > networkDomain information on the Nic to make it >>>> >>> >> > 3) SetNetworkACLCommand - creates network ACL on the virtual >>>> >>> router >>>> >>> >> > 4) SetSourceNatCommand - used for setting source nat on the >>>> Public >>>> >>> IP >>>> >>> >> on >>>> >>> >> > the VPC VR. >>>> >>> >> > 5) Site2SiteVpnCfgCommand - for setting up S2S VPN >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > Anthony/Kelven/Vijay did implementation for Xen/vmWare >>>> resources, >>>> >>> >> they can >>>> >>> >> > help you answering all hypervisor related questions. If you >>>> need >>>> >>> more >>>> >>> >> > details on business logic + Vpc VR management, I can help with >>>> >>> that. >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > -Alena. >>>> >>> >> > >>>> >>> >> >> >>>> >>> >> >>> -----Original Message----- >>>> >>> >> >>> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> >> >>> Sent: Wednesday, August 29, 2012 10:16 AM >>>> >>> >> >>> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> Subject: Re: VM router spawning multiple public nics >>>> >>> >> >>> >>>> >>> >> >>> I'd be willing to give it a shot if someone could point me >>>> in >>>> >>> the >>>> >>> >> >>> right direction and be available to answer questions. >>>> >>> >> >>> >>>> >>> >> >>> On Wed, Aug 29, 2012 at 10:39 AM, Edison Su >>>> >>> <edison...@citrix.com> >>>> >>> >> >>> wrote: >>>> >>> >> >>> > Yah, KVM doesn't support VPC yet. Will you help to add VPC >>>> >>> >> support on >>>> >>> >> >>> KVM?:) Just implement a few VPC related commands... >>>> >>> >> >>> > >>>> >>> >> >>> >> -----Original Message----- >>>> >>> >> >>> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> >> >>> >> Sent: Wednesday, August 29, 2012 6:49 AM >>>> >>> >> >>> >> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> >> Subject: Re: VM router spawning multiple public nics >>>> >>> >> >>> >> >>>> >>> >> >>> >> I can confirm that the patch has fixed my particular >>>> issue. >>>> >>> >> >>> >> >>>> >>> >> >>> >> This is likely unrelated and I think it doesn't even use >>>> the >>>> >>> >> same >>>> >>> >> >>> >> code, but I began to play with the VPC stuff a bit and >>>> >>> noticed >>>> >>> >> that >>>> >>> >> >>> I >>>> >>> >> >>> >> don't get any interfaces except for link local. I'd >>>> probably >>>> >>> >> chalk >>>> >>> >> >>> >> that up to it not being ready for KVM, but I thought it >>>> was >>>> >>> >> worth a >>>> >>> >> >>> >> mention. I'd be happy to try to help get it ready if >>>> someone >>>> >>> >> has >>>> >>> >> >>> time >>>> >>> >> >>> >> to nudge me in the right direction. >>>> >>> >> >>> >> >>>> >>> >> >>> >> On Tue, Aug 28, 2012 at 3:15 PM, Edison Su >>>> >>> >> <edison...@citrix.com> >>>> >>> >> >>> wrote: >>>> >>> >> >>> >> > >>>> >>> >> >>> >> > >>>> >>> >> >>> >> >> -----Original Message----- >>>> >>> >> >>> >> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> >> >>> >> >> Sent: Tuesday, August 28, 2012 2:00 PM >>>> >>> >> >>> >> >> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> >> >> Subject: Re: VM router spawning multiple public nics >>>> >>> >> >>> >> >> >>>> >>> >> >>> >> >> I thought about this solution myself, but below this >>>> >>> portion >>>> >>> >> of >>>> >>> >> >>> >> code >>>> >>> >> >>> >> >> it looks like it uses the hash map to determine which >>>> nic >>>> >>> >> number >>>> >>> >> >>> to >>>> >>> >> >>> >> >> add the IP to, so with multiple 'untagged' networks it >>>> >>> would >>>> >>> >> have >>>> >>> >> >>> no >>>> >>> >> >>> >> >> way of knowing which nicnum in the router corresponds >>>> with >>>> >>> >> the >>>> >>> >> >>> >> correct >>>> >>> >> >>> >> >> untagged vlan. >>>> >>> >> >>> >> >> >>>> >>> >> >>> >> >> nicNum = >>>> >>> >> vlanAllocatedToVM.get(ip.getVlanId()); >>>> >>> >> >>> >> >> networkUsage(routerIp, "addVif", "eth" >>>> + >>>> >>> >> nicNum); >>>> >>> >> >>> >> >> result = >>>> >>> >> >>> >> >> _virtRouterResource.assignPublicIpAddress(routerName, >>>> >>> >> >>> >> >> routerIp, ip.getPublicIp(), >>>> >>> >> ip.isAdd(), >>>> >>> >> >>> >> >> ip.isFirstIP(), >>>> >>> >> >>> >> >> ip.isSourceNat(), >>>> ip.getVlanId(), >>>> >>> >> >>> >> >> ip.getVlanGateway(), >>>> >>> >> >>> >> >> ip.getVlanNetmask(), >>>> >>> >> >>> ip.getVifMacAddress(), >>>> >>> >> >>> >> >> ip.getGuestIp(), nicNum); >>>> >>> >> >>> >> >> >>>> >>> >> >>> >> >> if ip.getVlanId() returns untagged (as it does on >>>> networks >>>> >>> >> with >>>> >>> >> >>> no >>>> >>> >> >>> >> >> vlan id), and we tried to put multiple untagged keys >>>> in >>>> >>> >> >>> >> >> vlanAllocatedToVM (as with multiple untagged networks), >>>> we >>>> >>> >> get >>>> >>> >> >>> the >>>> >>> >> >>> >> >> wrong nicNum, no? >>>> >>> >> >>> >> > >>>> >>> >> >>> >> > In the ipassoc case, if there are multiple untagged >>>> >>> networks, >>>> >>> >> all >>>> >>> >> >>> of >>>> >>> >> >>> >> them are use the same >>>> >>> >> >>> >> > Public bridge. Then multiple ip address will be added >>>> on >>>> >>> eth2 >>>> >>> >> >>> inside >>>> >>> >> >>> >> router VM. >>>> >>> >> >>> >> > If it works physically, then it works. >>>> >>> >> >>> >> > >>>> >>> >> >>> >> >> >>>> >>> >> >>> >> >> On Tue, Aug 28, 2012 at 2:48 PM, Edison Su >>>> >>> >> <edison...@citrix.com> >>>> >>> >> >>> >> wrote: >>>> >>> >> >>> >> >> > >>>> >>> >> >>> >> >> > >>>> >>> >> >>> >> >> >> -----Original Message----- >>>> >>> >> >>> >> >> >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>> >>> >> >>> >> >> >> Sent: Tuesday, August 28, 2012 1:40 PM >>>> >>> >> >>> >> >> >> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> >> >> >> Subject: Re: VM router spawning multiple public >>>> nics >>>> >>> >> >>> >> >> >> >>>> >>> >> >>> >> >> >> Yes, that looks like it would work for me, however >>>> >>> that's >>>> >>> >> not >>>> >>> >> >>> >> >> >> something that would ever make it into master, >>>> right? >>>> >>> >> >>> Essentially >>>> >>> >> >>> >> >> >> killing tagging for the public, private, and guest >>>> >>> traffic >>>> >>> >> >>> labels? >>>> >>> >> >>> >> >> >> There's also still the issue of not being able to >>>> >>> >> >>> differentiate >>>> >>> >> >>> >> >> >> between multiple untagged networks, if we wanted to >>>> add >>>> >>> an >>>> >>> >> IP >>>> >>> >> >>> to >>>> >>> >> >>> >> a >>>> >>> >> >>> >> >> >> router it might not know which untagged interface >>>> to >>>> >>> apply >>>> >>> >> it >>>> >>> >> >>> to. >>>> >>> >> >>> >> >> > >>>> >>> >> >>> >> >> > Physically, all the "untagged" network will be >>>> created >>>> >>> on >>>> >>> >> >>> >> >> public/guest/private bridge(the name we put in >>>> >>> >> >>> >> >> private/public/guest.bridge.name in agent.properties"). >>>> >>> >> >>> >> >> > Because, there is no way to create a new untagged >>>> bridge >>>> >>> by >>>> >>> >> >>> agent >>>> >>> >> >>> >> >> itself. Agent code only knows how to create a new >>>> >>> tagged(vlan) >>>> >>> >> >>> >> bridge. >>>> >>> >> >>> >> >> > So the fix should be pushed into master. >>>> >>> >> >>> >> >> > >>>> >>> >> >>> >> >> >> >>>> >>> >> >>> >> >> >> On Tue, Aug 28, 2012 at 2:32 PM, Edison Su >>>> >>> >> >>> <edison...@citrix.com> >>>> >>> >> >>> >> >> wrote: >>>> >>> >> >>> >> >> >> > >>>> >>> >> >>> >> >> >> > >>>> >>> >> >>> >> >> >> >> -----Original Message----- >>>> >>> >> >>> >> >> >> >> From: Marcus Sorensen >>>> [mailto:shadow...@gmail.com] >>>> >>> >> >>> >> >> >> >> Sent: Tuesday, August 28, 2012 12:23 PM >>>> >>> >> >>> >> >> >> >> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> >> >> >> >> Subject: Re: VM router spawning multiple public >>>> nics >>>> >>> >> >>> >> >> >> >> >>>> >>> >> >>> >> >> >> >> Thanks for pointing me in the right direction. >>>> I've >>>> >>> >> >>> reviewed >>>> >>> >> >>> >> this >>>> >>> >> >>> >> >> >> code >>>> >>> >> >>> >> >> >> >> in a bit more detail, and it seems like it's >>>> >>> >> accomplishing >>>> >>> >> >>> the >>>> >>> >> >>> >> >> >> >> following: >>>> >>> >> >>> >> >> >> >> >>>> >>> >> >>> >> >> >> >> 1. get all network interfaces currently >>>> connected to >>>> >>> >> the >>>> >>> >> >>> >> running >>>> >>> >> >>> >> >> VM >>>> >>> >> >>> >> >> >> >> (a.k.a vnet devices via libvirt) >>>> >>> >> >>> >> >> >> >> 2. find out which vlans these network interfaces >>>> are >>>> >>> >> >>> bridged >>>> >>> >> >>> >> to, >>>> >>> >> >>> >> >> >> store >>>> >>> >> >>> >> >> >> >> this in a hash map of vlan ids and nics >>>> >>> >> >>> >> >> >> >> 3. get all ip addresses to be added to the VM >>>> >>> >> >>> >> >> >> >> 4. for each ip, get the configured vlan id for >>>> the >>>> >>> ip, >>>> >>> >> >>> compare >>>> >>> >> >>> >> it >>>> >>> >> >>> >> >> to >>>> >>> >> >>> >> >> >> >> the hash map of existing vlan ids and nics >>>> >>> >> >>> >> >> >> >> 5. if the required vlan id is not found in the >>>> hash >>>> >>> map, >>>> >>> >> >>> >> create a >>>> >>> >> >>> >> >> >> new >>>> >>> >> >>> >> >> >> >> nic >>>> >>> >> >>> >> >> >> >> 6. assign the ip to the nic identified by the >>>> vlan >>>> >>> id >>>> >>> >> key >>>> >>> >> >>> in >>>> >>> >> >>> >> the >>>> >>> >> >>> >> >> >> hash >>>> >>> >> >>> >> >> >> >> map >>>> >>> >> >>> >> >> >> >> >>>> >>> >> >>> >> >> >> >> In this case, we're getting a vlan id returned >>>> in >>>> >>> step >>>> >>> >> 2 >>>> >>> >> >>> for a >>>> >>> >> >>> >> >> >> bridged >>>> >>> >> >>> >> >> >> >> nic whose network is defined as untagged in the >>>> >>> >> cloudstack >>>> >>> >> >>> db, >>>> >>> >> >>> >> >> >> >> therefore in step 5 we never match as already >>>> having >>>> >>> a >>>> >>> >> nic >>>> >>> >> >>> for >>>> >>> >> >>> >> >> >> >> 'untagged'. I wrote a big long response >>>> discussing >>>> >>> this >>>> >>> >> >>> issue, >>>> >>> >> >>> >> >> but >>>> >>> >> >>> >> >> >> as >>>> >>> >> >>> >> >> >> >> I began to dig further I realized that aside >>>> from my >>>> >>> >> >>> >> particular >>>> >>> >> >>> >> >> case, >>>> >>> >> >>> >> >> >> >> untagged vlans in general are just broken (for >>>> >>> example >>>> >>> >> they >>>> >>> >> >>> >> can't >>>> >>> >> >>> >> >> be >>>> >>> >> >>> >> >> >> >> dealt with uniquely in the current >>>> IpAssocCommand >>>> >>> code, >>>> >>> >> >>> given >>>> >>> >> >>> >> the >>>> >>> >> >>> >> >> >> hash >>>> >>> >> >>> >> >> >> >> map) and it would require more effort than I >>>> have >>>> >>> time >>>> >>> >> for >>>> >>> >> >>> now >>>> >>> >> >>> >> to >>>> >>> >> >>> >> >> >> make >>>> >>> >> >>> >> >> >> >> things work. If the code were already in place >>>> to >>>> >>> >> >>> >> differentiate >>>> >>> >> >>> >> >> >> >> between multiple untagged nics I think that >>>> fixing >>>> >>> my >>>> >>> >> >>> problem >>>> >>> >> >>> >> >> would >>>> >>> >> >>> >> >> >> be >>>> >>> >> >>> >> >> >> >> trivial, but since its not, I'll just find an >>>> >>> >> alternative >>>> >>> >> >>> >> >> solution. >>>> >>> >> >>> >> >> >> >> >>>> >>> >> >>> >> >> >> > >>>> >>> >> >>> >> >> >> > The untagged network usually means "untagged", no >>>> >>> vlan >>>> >>> >> on >>>> >>> >> >>> the >>>> >>> >> >>> >> >> >> bridge... >>>> >>> >> >>> >> >> >> > In your case, the untagged network actually has >>>> >>> >> vlan(tagged) >>>> >>> >> >>> on >>>> >>> >> >>> >> >> the >>>> >>> >> >>> >> >> >> bridge, thus getting things confused. >>>> >>> >> >>> >> >> >> > Will this patch(http://pastebin.com/HJXzZwKp) >>>> work >>>> >>> for >>>> >>> >> you? >>>> >>> >> >>> >> >> >> > >>>> >>> >> >>> >> >> >> >> On Mon, Aug 27, 2012 at 10:46 PM, Marcus >>>> Sorensen >>>> >>> >> >>> >> >> >> <shadow...@gmail.com> >>>> >>> >> >>> >> >> >> >> wrote: >>>> >>> >> >>> >> >> >> >> > ... >>>> >>> >> >>> >> >> >> >> > Integer nicPos = 0; >>>> >>> >> >>> >> >> >> >> > for (InterfaceDef nic : nics) { >>>> >>> >> >>> >> >> >> >> > if >>>> >>> >> >>> >> >> >> >> >>>> >>> (nic.getBrName().equalsIgnoreCase(_linkLocalBridgeName)) >>>> >>> >> { >>>> >>> >> >>> >> >> >> >> > >>>> >>> vlanAllocatedToVM.put("LinkLocal", >>>> >>> >> >>> >> nicPos); >>>> >>> >> >>> >> >> >> >> > } else { >>>> >>> >> >>> >> >> >> >> > String vlanId = >>>> >>> >> >>> >> >> >> >> getVlanIdFromBridge(nic.getBrName()); >>>> >>> >> >>> >> >> >> >> > if (vlanId != null) { >>>> >>> >> >>> >> >> >> >> > >>>> >>> vlanAllocatedToVM.put(vlanId, >>>> >>> >> >>> >> nicPos); >>>> >>> >> >>> >> >> >> >> > } else { >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> vlanAllocatedToVM.put(Vlan.UNTAGGED, >>>> >>> >> >>> >> >> >> nicPos); >>>> >>> >> >>> >> >> >> >> > } >>>> >>> >> >>> >> >> >> >> > } >>>> >>> >> >>> >> >> >> >> > nicPos++; >>>> >>> >> >>> >> >> >> >> > } >>>> >>> >> >>> >> >> >> >> > IpAddressTO[] ips = >>>> >>> cmd.getIpAddresses(); >>>> >>> >> >>> >> >> >> >> > int i = 0; >>>> >>> >> >>> >> >> >> >> > String result = null; >>>> >>> >> >>> >> >> >> >> > int nicNum = 0; >>>> >>> >> >>> >> >> >> >> > for (IpAddressTO ip : ips) { >>>> >>> >> >>> >> >> >> >> > if >>>> >>> >> >>> >> >> (!vlanAllocatedToVM.containsKey(ip.getVlanId())) >>>> >>> >> >>> >> >> >> { >>>> >>> >> >>> >> >> >> >> > /* plug a vif into router >>>> */ >>>> >>> >> >>> >> >> >> >> > VifHotPlug(conn, >>>> routerName, >>>> >>> >> >>> >> ip.getVlanId(), >>>> >>> >> >>> >> >> >> >> > >>>> ip.getVifMacAddress()); >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> vlanAllocatedToVM.put(ip.getVlanId(), >>>> >>> >> >>> >> >> >> nicPos++); >>>> >>> >> >>> >> >> >> >> > } >>>> >>> >> >>> >> >> >> >> > ... >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > Looks like the getVlanIdFromBridge might be a >>>> bit >>>> >>> >> >>> misleading. >>>> >>> >> >>> >> I >>>> >>> >> >>> >> >> am >>>> >>> >> >>> >> >> >> >> > running my guest public traffic on a >>>> 'cloudbr470', >>>> >>> >> which >>>> >>> >> >>> is >>>> >>> >> >>> >> a >>>> >>> >> >>> >> >> >> bridge >>>> >>> >> >>> >> >> >> >> > to eth2.470, yet I configured this network as >>>> >>> >> 'untagged' >>>> >>> >> >>> >> >> because I >>>> >>> >> >>> >> >> >> >> > have a vlan 470 available on eth3 for >>>> cloudstack >>>> >>> to >>>> >>> >> >>> >> autoassign >>>> >>> >> >>> >> >> >> (eth3 >>>> >>> >> >>> >> >> >> >> > is where all of my stuff will be autoassigned). >>>> So >>>> >>> >> I'm >>>> >>> >> >>> not >>>> >>> >> >>> >> 100% >>>> >>> >> >>> >> >> >> sure >>>> >>> >> >>> >> >> >> >> > yet what's going on here but it seems as >>>> though >>>> >>> the >>>> >>> >> above >>>> >>> >> >>> is >>>> >>> >> >>> >> >> not >>>> >>> >> >>> >> >> >> >> > setting any 'Vlan.UNTAGGED', since it finds a >>>> vlan >>>> >>> >> number >>>> >>> >> >>> >> for >>>> >>> >> >>> >> >> >> >> > eth2.470, but when it enumerates the IPs for >>>> the >>>> >>> >> router, >>>> >>> >> >>> it >>>> >>> >> >>> >> >> then >>>> >>> >> >>> >> >> >> runs >>>> >>> >> >>> >> >> >> >> > ip.getVlanId() and doesn't find a nic for the >>>> >>> >> untagged IP >>>> >>> >> >>> >> and >>>> >>> >> >>> >> >> >> creates >>>> >>> >> >>> >> >> >> >> > one. >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > I realize this is perhaps an uncommon case, >>>> but a >>>> >>> bug >>>> >>> >> >>> >> >> nonetheless. >>>> >>> >> >>> >> >> >> >> > I'll play with the code a bit and see if I can >>>> >>> come >>>> >>> >> up >>>> >>> >> >>> with >>>> >>> >> >>> >> a >>>> >>> >> >>> >> >> >> >> > solution. I'm thinking I can look at the nic's >>>> >>> >> broadcast >>>> >>> >> >>> URI >>>> >>> >> >>> >> >> and >>>> >>> >> >>> >> >> >> see >>>> >>> >> >>> >> >> >> >> > if it's supposed to be untagged, then add to >>>> >>> >> >>> >> vlanAllocatedToVM >>>> >>> >> >>> >> >> >> >> > appropriately, off the top of my head >>>> something >>>> >>> like: >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > String vlanId = >>>> >>> >> >>> >> >> >> >> getVlanIdFromBridge(nic.getBrName()); >>>> >>> >> >>> >> >> >> >> > if (vlanId != null && >>>> >>> >> >>> >> >> >> >> >>>> >>> > !nic.getBroadcastUri().toString().contains("untagged") >>>> >>> >> { >>>> >>> >> >>> >> >> >> >> > >>>> >>> vlanAllocatedToVM.put(vlanId, >>>> >>> >> >>> >> nicPos); >>>> >>> >> >>> >> >> >> >> > } else { >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> vlanAllocatedToVM.put(Vlan.UNTAGGED, >>>> >>> >> >>> >> >> >> nicPos); >>>> >>> >> >>> >> >> >> >> > } >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > >>>> >>> >> >>> >> >> >> >> > On Mon, Aug 27, 2012 at 6:42 PM, Edison Su >>>> >>> >> >>> >> >> <edison...@citrix.com> >>>> >>> >> >>> >> >> >> >> wrote: >>>> >>> >> >>> >> >> >> >> >> Possible bug in in kvm code: >>>> >>> >> LibvirtComputingResource- >>>> >>> >> >>> >> >> >> >> >execute(IpAssocCommand cmd)-> VifHotPlug, which >>>> is >>>> >>> >> only >>>> >>> >> >>> place >>>> >>> >> >>> >> >> >> adding >>>> >>> >> >>> >> >> >> >> nic into router vm. >>>> >>> >> >>> >> >> >> >> >> Turn on agent log, then take a look what >>>> happened. >>>> >>> >> >>> >> >> >> >> >> >>>> >>> >> >>> >> >> >> >> >>> -----Original Message----- >>>> >>> >> >>> >> >> >> >> >>> From: Marcus Sorensen >>>> >>> [mailto:shadow...@gmail.com] >>>> >>> >> >>> >> >> >> >> >>> Sent: Monday, August 27, 2012 5:10 PM >>>> >>> >> >>> >> >> >> >> >>> To: cloudstack-dev@incubator.apache.org >>>> >>> >> >>> >> >> >> >> >>> Subject: VM router spawning multiple public >>>> nics >>>> >>> >> >>> >> >> >> >> >>> >>>> >>> >> >>> >> >> >> >> >>> I've got two zones running the same build of >>>> >>> >> cloudstack >>>> >>> >> >>> (a >>>> >>> >> >>> >> >> >> recent >>>> >>> >> >>> >> >> >> >> copy >>>> >>> >> >>> >> >> >> >> >>> of master). One of them creates routers that >>>> >>> turn >>>> >>> >> into >>>> >>> >> >>> >> ugly >>>> >>> >> >>> >> >> >> >> >>> multi-headed beasts, and by that I mean that >>>> any >>>> >>> >> time I >>>> >>> >> >>> >> >> create a >>>> >>> >> >>> >> >> >> >> port >>>> >>> >> >>> >> >> >> >> >>> forwarding or iptables rule for that router >>>> I >>>> >>> get a >>>> >>> >> new >>>> >>> >> >>> >> >> public >>>> >>> >> >>> >> >> >> NIC >>>> >>> >> >>> >> >> >> >> >>> with an identical IP address, I have an >>>> instance >>>> >>> >> with a >>>> >>> >> >>> >> few >>>> >>> >> >>> >> >> tens >>>> >>> >> >>> >> >> >> of >>>> >>> >> >>> >> >> >> >> >>> NICs. My guess is that some script isn't >>>> >>> detecting >>>> >>> >> >>> that >>>> >>> >> >>> >> >> there's >>>> >>> >> >>> >> >> >> >> >>> already a NIC with the public IP on it. It >>>> >>> looks >>>> >>> >> fine >>>> >>> >> >>> in >>>> >>> >> >>> >> the >>>> >>> >> >>> >> >> >> >> >>> database, there is only one public NIC >>>> defined >>>> >>> in >>>> >>> >> the >>>> >>> >> >>> nics >>>> >>> >> >>> >> >> table. >>>> >>> >> >>> >> >> >> >> >>> I'll troubleshoot it tomorrow, but if anyone >>>> >>> knows >>>> >>> >> >>> where I >>>> >>> >> >>> >> >> >> should >>>> >>> >> >>> >> >> >> >> >>> begin the headstart would be appreciated. >>>> >>> >> >>> >> >> >> >> >>> >>>> >>> >> >>> >> >> >> >> >>> Thanks >>>> >>> >> >> >>>> >>> >> > >>>> >>> >> >
