Would implementing a VPC Redundant Router Network Guru using the an extension of the NetworkGuru interface be an appropriate way to create a VPC network with redundant routers. It seems that extending Network Guru interface or extending one of it's children eg the abstract class GuestNetworkGuru would be a appropriate design path.
Comments? Karl On Mon, Feb 3, 2014 at 2:58 PM, Daan Hoogland <daan.hoogl...@gmail.com>wrote: > On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sh...@yasker.org> wrote: > > On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.har...@sungard.com > >wrote: > > > >> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogl...@gmail.com > >> >wrote: > >> > >> > > >> > On 03 Feb 2014, at 19:45, Karl Harris <karl.har...@sungard.com> > wrote: > >> > > >> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland < > >> daan.hoogl...@gmail.com > >> > >wrote: > >> > > > >> > >> > >> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.har...@sungard.com> > >> wrote: > >> > >> > >> > >>> After discussion with my colleagues questions about initial > >> > >>> configuration of the open network redundant routers and the > >> > >>> applicability of the existing bash scripts (cloud-early-config) to > >> the > >> > >>> setup of VPC redundant routers have generated. > >> > >>> > >> > >>> > >> > >>> Some setup first: In the bash script cloud-early-config there is a > >> > >>> function named setup_redundant_router which makes copies of > several > >> > >>> template files. The template files are used to configure > keepalived > >> > >>> and conntrackd. The template copies are edited, via sed editor, > using > >> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.) which > are > >> > >>> obtained from the kernel of the current running linux image using > the > >> > >>> virtual file system /proc/cmdline. > >> > >>> > >> > >>> I'm sure keepalived and conntrackd can be used for starting and > >> > >>> control of VPC redundant routers. However the setup of keepalived > and > >> > >>> conntrackd for VPC needs setup parameters which are dynamic > because a > >> > >>> VPC can have N number of redundant router pairs, not just the > fixed > >> > >>> number parsed from proc/cmdline in the running kernel. > >> > >>> > >> > >>> Am I correct in this analysis? > >> > >> Karl, I think not. There is only one router in a vac, it routes for > >> all > >> > >> networks in the virtual private cloud. Am I misreading your > >> description. > >> > > > >> > > Ok then a VPC should contain a single router containing N+1 (N+2 if > >> > using a > >> > > VPN connection) nic's? Where N is the number of private networks. > >> > And I would like to see N+3 as it will allocate another when a private > >> > gateway is defined. That is why I proposed to pre-allocate this nic > to be > >> > able to predict mic-ids, i.e. eth<#>. > >> > > >> > > > >> > > If so then my original question still holds in the sense that the > nic's > >> > > need to be created in the VM kernel based on the number of private > >> > networks > >> > > desired? > >> > > >> > Yes > >> > > >> > > I found the method update_System_Vm_Templates in the jar file > >> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao > which > >> > > seems to point to a set of pre-configured VM images. > >> > I can not find the method you describe but these preconfigured images > are > >> > the ones used for all the system vms. That would be secondary storage > >> vms, > >> > console proxy vms and indeed the router vms. > >> > > >> > Based on the discussion above the the configuration part of the VPC > >> redundant router is: > >> > >> 1. Collecting the correct data required to configure the nics. (Is this > >> part already in the VPC setup code? I will look for it, pointers will be > >> helpful.) > >> > > > > vpc_guestgw.sh > and VpcVirtualNetworkApplianceManager and -Impl and their ancestors > > > > > > > >> 2. Based on the hypervisor/vm types configure the nic(s) on the router > VM > >> using the data collected above (Same question as in 1 above is this code > >> already in place?). My guess is to start with a preconfigured template > and > > > > configure the vm based on the number of nics and any other appropriate > data. > >> > > > > Yes. > > > > > >> 3. Configuring conntrackd and keepalived after the router vm (nics) > have > >> been configured and the router(s) started or restarted? I still need to > >> work through this. > >> > > > > Likely you would need this. > > > > Don't know if keepalived can be start if there is no virtual IP > definition. > > > > You also need to think about what to do second router(or the router pair) > > when the last tier of VPC has been destroyed. > > > > --Sheng > > > > > >> > >> > >> > >> > >> > > >> > >> > >> > >>> If so, given the dynamic nature of the VPC redundant router > >> > >>> configurations: Is using a setup_VPC_redundant_router bash > function, > >> > >>> similar to the existing open network function mentioned above, the > >> > >>> most appropriate way to setup the keepalived or conntrackd > >> > >>> configuration files for VPC redundant routers in the > >> > >>> cloud-early-script? It seems to me reading the parameters from the > >> > >>> kernel will require a unwieldy set of kernels to match the N > private > >> > >>> network redundant router pairs configured by the enduser. > >> > >>> > >> > >>> > >> > >>> Comments, questions, clarifications? > >> > >> > >> > >>> > >> > >>> Karl > >> > >>> > >> > >>> > >> > >>> In the bash script ea > >> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland < > >> > daan.hoogl...@gmail.com> > >> > >> wrote: > >> > >>>> good reason to skip it for a next version, let's look into it > >> anyway, > >> > >>>> as we don't want to burn any of our ships. > >> > >>>> > >> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris < > >> karl.har...@sungard.com > >> > > > >> > >> wrote: > >> > >>>>> All, > >> > >>>>> > >> > >>>>> At first redundant DHCP seemed like a good idea. I did some > cursory > >> > >>>>> research and the more I read the more I'm convinced it may be > >> > >>>>> more trouble than its worth for the first implementation. I'll > talk > >> > >>>>> with some of our Systems Engineer's here and get a broader > >> > >>>>> perspective. > >> > >>>>> > >> > >>>>> There seems to be only a single implementation of an open source > >> DHCP > >> > >>>>> server that will handle the synchronization required for > redundant > >> > >>>>> servers. > >> > >>>>> > >> > >>>>> Karl > >> > >>>>> > >> > >>>>> > >> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland < > >> > >> daan.hoogl...@gmail.com> wrote: > >> > >>>>>> Saurav, > >> > >>>>>> > >> > >>>>>> Not sure how this happens now, but it is definateluy something > we > >> > >>>>>> want. For the static conf files it won't be much of a problem. > The > >> > >>>>>> firewall/loadbalences won't be much of a problem, they are fire > >> and > >> > >>>>>> forget configurations of the ms that can just be sent to both. > The > >> > >>>>>> dhcp is a challange. I am not sure if it is solved for the > plain > >> rvr > >> > >>>>>> now but it should be solved for that as well. > >> > >>>>>> > >> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri > >> > >>>>>> <saurav.lah...@sungard.com> wrote: > >> > >>>>>>> Daan, > >> > >>>>>>> I was wondering if you had not shared your thoughts, but looks > >> like > >> > >> I had > >> > >>>>>>> missed your mail. > >> > >>>>>>> > >> > >>>>>>> I agree that redundant dhcp or dns would be good to have. > What I > >> > >> meant was, > >> > >>>>>>> it appears that by just enabling RVR there is no way to > auto > >> > sync > >> > >>>>>>> configuration between the master and slave nodes with regard > to > >> > >> dhcp, > >> > >>>>>>> loadbalancer and firewall(specifically the dhcp lease file, > >> > >> haproxy,cfg and > >> > >>>>>>> iptables configuration). So just enabling RVR does not ensure > >> high > >> > >>>>>>> availability for these services. Is there a way cloudstack > >> > autosyncs > >> > >>>>>>> configuration? > >> > >>>>>>> > >> > >>>>>>> For the routing portion this is not an issue as the > participating > >> > >> routers > >> > >>>>>>> learn the route through known protocols. > >> > >>>>>>> > >> > >>>>>>> Thanks > >> > >>>>>>> Saurav > >> > >>>>>>> > >> > >>>>>>> > >> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland < > >> > >> daan.hoogl...@gmail.com>wrote: > >> > >>>>>>> > >> > >>>>>>>> Saurav, I don't see why you can't benefit from having other > >> > services > >> > >>>>>>>> redundant as well. Vpn might be a problem as the source ip > of a > >> > >>>>>>>> redundant router pair on a vpn might give a problem (maybe > there > >> > is > >> > >> an > >> > >>>>>>>> implementation) but why wouldn't you want redundant dhcp or > dns > >> > >>>>>>>> services? As I understand it these are used at Schuberg > Philis > >> at > >> > >> the > >> > >>>>>>>> moment. will double check when I get a chance. > >> > >>>>>>>> > >> > >>>>>>>> regards, > >> > >>>>>>>> Daan > >> > >>>>>>>> > >> > >>>>>>>> > >> > >>>>>>>> > >> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav Lahiri > >> > >>>>>>>> <saurav.lah...@sungard.com> wrote: > >> > >>>>>>>>> Daan, > >> > >>>>>>>>> So looking at what is available today for guest network, the > >> > >> Redundant VR > >> > >>>>>>>>> can be enabled only for the source nat service. I guess the > >> > >> mention of > >> > >>>>>>>> the > >> > >>>>>>>>> source nat router is in relation to that. I could be wrong > >> > though. > >> > >> It > >> > >>>>>>>>> appears that the other services like vpn, dhcp, dns do not > >> > >> benefit much > >> > >>>>>>>>> from the RVR capability. Can you clarify if thats correct? > >> > >>>>>>>>> > >> > >>>>>>>>> Thanks > >> > >>>>>>>>> Saurav > >> > >>>>>>>>> > >> > >>>>>>>>> > >> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl Harris < > >> > >> karl.har...@sungard.com > >> > >>>>>>>>> wrote: > >> > >>>>>>>>> > >> > >>>>>>>>>> ---------- Forwarded message ---------- > >> > >>>>>>>>>> From: Daan Hoogland <daan.hoogl...@gmail.com> > >> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51 PM > >> > >>>>>>>>>> Subject: Re: rvr4vpc > >> > >>>>>>>>>> To: Karl Harris <karl.har...@sungard.com> > >> > >>>>>>>>>> Cc: Christopher Litsinger < > christopher.litsi...@sungard.com> > >> > >>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> H Karl, > >> > >>>>>>>>>> > >> > >>>>>>>>>> Thanks for sharing. I didn't want to initiate this but I > >> > >> encourage you > >> > >>>>>>>>>> to share this on the dev list (not in jira) as things are > only > >> > >>>>>>>>>> considered 'discussed' if they passed by there. > >> > >>>>>>>>>> > >> > >>>>>>>>>> You speak of '1 Get configuration data on Source Nat > Router', > >> I > >> > >> don't > >> > >>>>>>>>>> understand why you call the router by this designation. > >> 'Source > >> > >> Nat' > >> > >>>>>>>>>> is only one of it's many possible functions. > >> > >>>>>>>>>> > >> > >>>>>>>>>> Apart from the design principles I shared with you I have > so > >> far > >> > >> only > >> > >>>>>>>>>> a technical implementation detail so far. That is to > reserve > >> the > >> > >>>>>>>>>> (eth2) interface for the private gateway on the vpc (r)vr. > >> This > >> > >> way > >> > >>>>>>>>>> the inteface to configure are somewhat predictable. > >> > >>>>>>>>>> > >> > >>>>>>>>>> As for the design principle to have a statefull router > (reboot > >> > >> proof) > >> > >>>>>>>>>> the idea is to implement a configuration file that will be > >> > >> uploaded to > >> > >>>>>>>>>> the router after which a self-config command is send that > will > >> > >>>>>>>>>> implement the details of configuring the interfaces, > haproxy > >> and > >> > >>>>>>>>>> keepalived and maybe more. I think your current assessment > of > >> > the > >> > >>>>>>>>>> working of the RVRs is accurate but it will not be workable > >> for > >> > an > >> > >>>>>>>>>> implementation for vpc's as they have an unpreditable > number > >> of > >> > >>>>>>>>>> interfaces. > >> > >>>>>>>>>> > >> > >>>>>>>>>> to bad you can't make it next thursday, > >> > >>>>>>>>>> Daan Hoogland > >> > >>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM, Karl Harris < > >> > >> karl.har...@sungard.com> > >> > >>>>>>>>>> wrote: > >> > >>>>>>>>>>> Daan, > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> Sorry for the delayed response but as Christopher > mentioned > >> to > >> > >> you in > >> > >>>>>>>> his > >> > >>>>>>>>>>> email I am getting my head around the CloudStack software. > >> > >>>>>>>>>>> Since I am new to CloudStack but "old" to enterprise level > >> JAVA > >> > >> the > >> > >>>>>>>> task > >> > >>>>>>>>>> is > >> > >>>>>>>>>>> large but not impossible. I have no experience with > running > >> > >> CloudStack > >> > >>>>>>>>>> but > >> > >>>>>>>>>>> considerable experience designing and maintaining large > JAVA > >> > >>>>>>>>>> applications. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> I've created what I believe is a very high level abstract > of > >> > how > >> > >> the > >> > >>>>>>>>>> current > >> > >>>>>>>>>>> guest VRR's are created for guest networks with the > intent of > >> > >> making > >> > >>>>>>>> this > >> > >>>>>>>>>>> abstract > >> > >>>>>>>>>>> more detailed. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> 1 Get configuration data on Source Nat Router selected > as a > >> > >> redundant > >> > >>>>>>>>>>> router > >> > >>>>>>>>>>> 1.1 Public and Guest network identified. > >> > >>>>>>>>>>> 2 Both routers are provisioned > >> > >>>>>>>>>>> 2.1 Software trys different, > >> > >> regions(?),zones,pods,clusters,hosts > >> > >>>>>>>> in > >> > >>>>>>>>>>> that order as the location of the router. Log maximum > >> > "distance" > >> > >>>>>>>> apart. > >> > >>>>>>>>>>> 3 Keepalived is configured > >> > >>>>>>>>>>> 4 Both routers are started > >> > >>>>>>>>>>> 5 Keepalived is started > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> Obviously there is much more that is happening under each > of > >> > the > >> > >> steps > >> > >>>>>>>>>>> above. My intent is to complete this detailed "as is" > listing > >> > as > >> > >> much > >> > >>>>>>>> as > >> > >>>>>>>>>> we > >> > >>>>>>>>>>> can. Then using the "as is" description/sequence > >> > >>>>>>>>>>> make a "to-be" addition for VPC's. When I get a consensus > on > >> > WHAT > >> > >>>>>>>> needs > >> > >>>>>>>>>> to > >> > >>>>>>>>>>> be implemented for the VRR in VPC then develop HOW best > to > >> > >> implement > >> > >>>>>>>> the > >> > >>>>>>>>>>> "to-be" addition with the > >> > >>>>>>>>>>> existing JAVA code as well as what additional classes > need to > >> > be > >> > >>>>>>>>>>> extended/implemented/created. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> Comments, critiques and changes to the above sequence are > >> > >> encouraged. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> I plan on posting this to the dev-list/Jira very soon. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> I have been using this functional spec as a guide, after > >> > >> discussing > >> > >>>>>>>> this > >> > >>>>>>>>>>> with our Systems Engineering people this spec meets our > >> > >> requirements. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> Do you have an implementation in mind? > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> We have an internal Cloud Meeting with conflicts with the > >> > >> cloudstack > >> > >>>>>>>>>> "day" > >> > >>>>>>>>>>> next week so I will not be in attendence. > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> Karl > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> On Tue, Jan 14, 2014 at 4:35 PM, Daan Hoogland < > >> > >>>>>>>> daan.hoogl...@gmail.com> > >> > >>>>>>>>>>> wrote: > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>>> Hello overthere in the states, > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>>> Tomorow I will start some experimenting with redundant > vpc > >> > >> routers. > >> > >>>>>>>>>>>> This is to check up on any findings and requirements that > >> you > >> > >> might > >> > >>>>>>>>>>>> have on this. Once again I would not like to waste work > on > >> > this > >> > >> as it > >> > >>>>>>>>>>>> is really a globally usable feature that is probably > >> > universal. > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>>> please let me know your status on this. > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>>> If any of you are coming to the cloudstack day in London > >> next > >> > >> week, > >> > >>>>>>>>>>>> let's meetup next thursday. > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>>> kind regards, > >> > >>>>>>>>>>>> Daan Hoogland > >> > >>>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> -- > >> > >>>>>>>>>>> Karl O. Harris > >> > >>>>>>>>>>> Cloud Software Engineer > >> > >>>>>>>>>>> Sungard Availability Services > >> > >>>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> > >> > >>>>>>>>>> -- > >> > >>>>>>>>>> Karl O. Harris > >> > >>>>>>>>>> Cloud Software Engineer > >> > >>>>>>>>>> Sungard Availability Services > >> > >>>>>>>>>> > >> > >>>>>>>> > >> > >>>>>>>> > >> > >>>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> -- > >> > >>>>> Karl O. Harris > >> > >>>>> Cloud Software Engineer > >> > >>>>> Sungard Availability Services > >> > >>>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> -- > >> > >>> Karl O. Harris > >> > >>> Cloud Software Engineer > >> > >>> Sungard Availability Services > >> > >> > >> > >> > >> > >> > >> > > > >> > > > >> > > -- > >> > > Karl O. Harris > >> > > Cloud Software Engineer > >> > > Sungard Availability Services > >> > > >> > > >> > >> > >> -- > >> Karl O. Harris > >> Cloud Software Engineer > >> Sungard Availability Services > >> > > > > -- > Daan > > -- Karl O. Harris Cloud Software Engineer Sungard Availability Services
