On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sh...@yasker.org> wrote: > On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.har...@sungard.com>wrote: > >> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogl...@gmail.com >> >wrote: >> >> > >> > On 03 Feb 2014, at 19:45, Karl Harris <karl.har...@sungard.com> wrote: >> > >> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland < >> daan.hoogl...@gmail.com >> > >wrote: >> > > >> > >> >> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.har...@sungard.com> >> wrote: >> > >> >> > >>> After discussion with my colleagues questions about initial >> > >>> configuration of the open network redundant routers and the >> > >>> applicability of the existing bash scripts (cloud-early-config) to >> the >> > >>> setup of VPC redundant routers have generated. >> > >>> >> > >>> >> > >>> Some setup first: In the bash script cloud-early-config there is a >> > >>> function named setup_redundant_router which makes copies of several >> > >>> template files. The template files are used to configure keepalived >> > >>> and conntrackd. The template copies are edited, via sed editor, using >> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.) which are >> > >>> obtained from the kernel of the current running linux image using the >> > >>> virtual file system /proc/cmdline. >> > >>> >> > >>> I'm sure keepalived and conntrackd can be used for starting and >> > >>> control of VPC redundant routers. However the setup of keepalived and >> > >>> conntrackd for VPC needs setup parameters which are dynamic because a >> > >>> VPC can have N number of redundant router pairs, not just the fixed >> > >>> number parsed from proc/cmdline in the running kernel. >> > >>> >> > >>> Am I correct in this analysis? >> > >> Karl, I think not. There is only one router in a vac, it routes for >> all >> > >> networks in the virtual private cloud. Am I misreading your >> description. >> > > >> > > Ok then a VPC should contain a single router containing N+1 (N+2 if >> > using a >> > > VPN connection) nic's? Where N is the number of private networks. >> > And I would like to see N+3 as it will allocate another when a private >> > gateway is defined. That is why I proposed to pre-allocate this nic to be >> > able to predict mic-ids, i.e. eth<#>. >> > >> > > >> > > If so then my original question still holds in the sense that the nic's >> > > need to be created in the VM kernel based on the number of private >> > networks >> > > desired? >> > >> > Yes >> > >> > > I found the method update_System_Vm_Templates in the jar file >> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao which >> > > seems to point to a set of pre-configured VM images. >> > I can not find the method you describe but these preconfigured images are >> > the ones used for all the system vms. That would be secondary storage >> vms, >> > console proxy vms and indeed the router vms. >> > >> > Based on the discussion above the the configuration part of the VPC >> redundant router is: >> >> 1. Collecting the correct data required to configure the nics. (Is this >> part already in the VPC setup code? I will look for it, pointers will be >> helpful.) >> > > vpc_guestgw.sh and VpcVirtualNetworkApplianceManager and -Impl and their ancestors
> > >> 2. Based on the hypervisor/vm types configure the nic(s) on the router VM >> using the data collected above (Same question as in 1 above is this code >> already in place?). My guess is to start with a preconfigured template and > > configure the vm based on the number of nics and any other appropriate data. >> > > Yes. > > >> 3. Configuring conntrackd and keepalived after the router vm (nics) have >> been configured and the router(s) started or restarted? I still need to >> work through this. >> > > Likely you would need this. > > Don't know if keepalived can be start if there is no virtual IP definition. > > You also need to think about what to do second router(or the router pair) > when the last tier of VPC has been destroyed. > > --Sheng > > >> >> >> >> >> > >> > >> >> > >>> If so, given the dynamic nature of the VPC redundant router >> > >>> configurations: Is using a setup_VPC_redundant_router bash function, >> > >>> similar to the existing open network function mentioned above, the >> > >>> most appropriate way to setup the keepalived or conntrackd >> > >>> configuration files for VPC redundant routers in the >> > >>> cloud-early-script? It seems to me reading the parameters from the >> > >>> kernel will require a unwieldy set of kernels to match the N private >> > >>> network redundant router pairs configured by the enduser. >> > >>> >> > >>> >> > >>> Comments, questions, clarifications? >> > >> >> > >>> >> > >>> Karl >> > >>> >> > >>> >> > >>> In the bash script ea >> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland < >> > daan.hoogl...@gmail.com> >> > >> wrote: >> > >>>> good reason to skip it for a next version, let's look into it >> anyway, >> > >>>> as we don't want to burn any of our ships. >> > >>>> >> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris < >> karl.har...@sungard.com >> > > >> > >> wrote: >> > >>>>> All, >> > >>>>> >> > >>>>> At first redundant DHCP seemed like a good idea. I did some cursory >> > >>>>> research and the more I read the more I'm convinced it may be >> > >>>>> more trouble than its worth for the first implementation. I'll talk >> > >>>>> with some of our Systems Engineer's here and get a broader >> > >>>>> perspective. >> > >>>>> >> > >>>>> There seems to be only a single implementation of an open source >> DHCP >> > >>>>> server that will handle the synchronization required for redundant >> > >>>>> servers. >> > >>>>> >> > >>>>> Karl >> > >>>>> >> > >>>>> >> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland < >> > >> daan.hoogl...@gmail.com> wrote: >> > >>>>>> Saurav, >> > >>>>>> >> > >>>>>> Not sure how this happens now, but it is definateluy something we >> > >>>>>> want. For the static conf files it won't be much of a problem. The >> > >>>>>> firewall/loadbalences won't be much of a problem, they are fire >> and >> > >>>>>> forget configurations of the ms that can just be sent to both. The >> > >>>>>> dhcp is a challange. I am not sure if it is solved for the plain >> rvr >> > >>>>>> now but it should be solved for that as well. >> > >>>>>> >> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri >> > >>>>>> <saurav.lah...@sungard.com> wrote: >> > >>>>>>> Daan, >> > >>>>>>> I was wondering if you had not shared your thoughts, but looks >> like >> > >> I had >> > >>>>>>> missed your mail. >> > >>>>>>> >> > >>>>>>> I agree that redundant dhcp or dns would be good to have. What I >> > >> meant was, >> > >>>>>>> it appears that by just enabling RVR there is no way to auto >> > sync >> > >>>>>>> configuration between the master and slave nodes with regard to >> > >> dhcp, >> > >>>>>>> loadbalancer and firewall(specifically the dhcp lease file, >> > >> haproxy,cfg and >> > >>>>>>> iptables configuration). So just enabling RVR does not ensure >> high >> > >>>>>>> availability for these services. Is there a way cloudstack >> > autosyncs >> > >>>>>>> configuration? >> > >>>>>>> >> > >>>>>>> For the routing portion this is not an issue as the participating >> > >> routers >> > >>>>>>> learn the route through known protocols. >> > >>>>>>> >> > >>>>>>> Thanks >> > >>>>>>> Saurav >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland < >> > >> daan.hoogl...@gmail.com>wrote: >> > >>>>>>> >> > >>>>>>>> Saurav, I don't see why you can't benefit from having other >> > services >> > >>>>>>>> redundant as well. Vpn might be a problem as the source ip of a >> > >>>>>>>> redundant router pair on a vpn might give a problem (maybe there >> > is >> > >> an >> > >>>>>>>> implementation) but why wouldn't you want redundant dhcp or dns >> > >>>>>>>> services? As I understand it these are used at Schuberg Philis >> at >> > >> the >> > >>>>>>>> moment. will double check when I get a chance. >> > >>>>>>>> >> > >>>>>>>> regards, >> > >>>>>>>> Daan >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav Lahiri >> > >>>>>>>> <saurav.lah...@sungard.com> wrote: >> > >>>>>>>>> Daan, >> > >>>>>>>>> So looking at what is available today for guest network, the >> > >> Redundant VR >> > >>>>>>>>> can be enabled only for the source nat service. I guess the >> > >> mention of >> > >>>>>>>> the >> > >>>>>>>>> source nat router is in relation to that. I could be wrong >> > though. >> > >> It >> > >>>>>>>>> appears that the other services like vpn, dhcp, dns do not >> > >> benefit much >> > >>>>>>>>> from the RVR capability. Can you clarify if thats correct? >> > >>>>>>>>> >> > >>>>>>>>> Thanks >> > >>>>>>>>> Saurav >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl Harris < >> > >> karl.har...@sungard.com >> > >>>>>>>>> wrote: >> > >>>>>>>>> >> > >>>>>>>>>> ---------- Forwarded message ---------- >> > >>>>>>>>>> From: Daan Hoogland <daan.hoogl...@gmail.com> >> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51 PM >> > >>>>>>>>>> Subject: Re: rvr4vpc >> > >>>>>>>>>> To: Karl Harris <karl.har...@sungard.com> >> > >>>>>>>>>> Cc: Christopher Litsinger <christopher.litsi...@sungard.com> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> H Karl, >> > >>>>>>>>>> >> > >>>>>>>>>> Thanks for sharing. I didn't want to initiate this but I >> > >> encourage you >> > >>>>>>>>>> to share this on the dev list (not in jira) as things are only >> > >>>>>>>>>> considered 'discussed' if they passed by there. >> > >>>>>>>>>> >> > >>>>>>>>>> You speak of '1 Get configuration data on Source Nat Router', >> I >> > >> don't >> > >>>>>>>>>> understand why you call the router by this designation. >> 'Source >> > >> Nat' >> > >>>>>>>>>> is only one of it's many possible functions. >> > >>>>>>>>>> >> > >>>>>>>>>> Apart from the design principles I shared with you I have so >> far >> > >> only >> > >>>>>>>>>> a technical implementation detail so far. That is to reserve >> the >> > >>>>>>>>>> (eth2) interface for the private gateway on the vpc (r)vr. >> This >> > >> way >> > >>>>>>>>>> the inteface to configure are somewhat predictable. >> > >>>>>>>>>> >> > >>>>>>>>>> As for the design principle to have a statefull router (reboot >> > >> proof) >> > >>>>>>>>>> the idea is to implement a configuration file that will be >> > >> uploaded to >> > >>>>>>>>>> the router after which a self-config command is send that will >> > >>>>>>>>>> implement the details of configuring the interfaces, haproxy >> and >> > >>>>>>>>>> keepalived and maybe more. I think your current assessment of >> > the >> > >>>>>>>>>> working of the RVRs is accurate but it will not be workable >> for >> > an >> > >>>>>>>>>> implementation for vpc's as they have an unpreditable number >> of >> > >>>>>>>>>> interfaces. >> > >>>>>>>>>> >> > >>>>>>>>>> to bad you can't make it next thursday, >> > >>>>>>>>>> Daan Hoogland >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM, Karl Harris < >> > >> karl.har...@sungard.com> >> > >>>>>>>>>> wrote: >> > >>>>>>>>>>> Daan, >> > >>>>>>>>>>> >> > >>>>>>>>>>> Sorry for the delayed response but as Christopher mentioned >> to >> > >> you in >> > >>>>>>>> his >> > >>>>>>>>>>> email I am getting my head around the CloudStack software. >> > >>>>>>>>>>> Since I am new to CloudStack but "old" to enterprise level >> JAVA >> > >> the >> > >>>>>>>> task >> > >>>>>>>>>> is >> > >>>>>>>>>>> large but not impossible. I have no experience with running >> > >> CloudStack >> > >>>>>>>>>> but >> > >>>>>>>>>>> considerable experience designing and maintaining large JAVA >> > >>>>>>>>>> applications. >> > >>>>>>>>>>> >> > >>>>>>>>>>> I've created what I believe is a very high level abstract of >> > how >> > >> the >> > >>>>>>>>>> current >> > >>>>>>>>>>> guest VRR's are created for guest networks with the intent of >> > >> making >> > >>>>>>>> this >> > >>>>>>>>>>> abstract >> > >>>>>>>>>>> more detailed. >> > >>>>>>>>>>> >> > >>>>>>>>>>> 1 Get configuration data on Source Nat Router selected as a >> > >> redundant >> > >>>>>>>>>>> router >> > >>>>>>>>>>> 1.1 Public and Guest network identified. >> > >>>>>>>>>>> 2 Both routers are provisioned >> > >>>>>>>>>>> 2.1 Software trys different, >> > >> regions(?),zones,pods,clusters,hosts >> > >>>>>>>> in >> > >>>>>>>>>>> that order as the location of the router. Log maximum >> > "distance" >> > >>>>>>>> apart. >> > >>>>>>>>>>> 3 Keepalived is configured >> > >>>>>>>>>>> 4 Both routers are started >> > >>>>>>>>>>> 5 Keepalived is started >> > >>>>>>>>>>> >> > >>>>>>>>>>> Obviously there is much more that is happening under each of >> > the >> > >> steps >> > >>>>>>>>>>> above. My intent is to complete this detailed "as is" listing >> > as >> > >> much >> > >>>>>>>> as >> > >>>>>>>>>> we >> > >>>>>>>>>>> can. Then using the "as is" description/sequence >> > >>>>>>>>>>> make a "to-be" addition for VPC's. When I get a consensus on >> > WHAT >> > >>>>>>>> needs >> > >>>>>>>>>> to >> > >>>>>>>>>>> be implemented for the VRR in VPC then develop HOW best to >> > >> implement >> > >>>>>>>> the >> > >>>>>>>>>>> "to-be" addition with the >> > >>>>>>>>>>> existing JAVA code as well as what additional classes need to >> > be >> > >>>>>>>>>>> extended/implemented/created. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Comments, critiques and changes to the above sequence are >> > >> encouraged. >> > >>>>>>>>>>> >> > >>>>>>>>>>> I plan on posting this to the dev-list/Jira very soon. >> > >>>>>>>>>>> >> > >>>>>>>>>>> I have been using this functional spec as a guide, after >> > >> discussing >> > >>>>>>>> this >> > >>>>>>>>>>> with our Systems Engineering people this spec meets our >> > >> requirements. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Do you have an implementation in mind? >> > >>>>>>>>>>> >> > >>>>>>>>>>> We have an internal Cloud Meeting with conflicts with the >> > >> cloudstack >> > >>>>>>>>>> "day" >> > >>>>>>>>>>> next week so I will not be in attendence. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Karl >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> On Tue, Jan 14, 2014 at 4:35 PM, Daan Hoogland < >> > >>>>>>>> daan.hoogl...@gmail.com> >> > >>>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Hello overthere in the states, >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Tomorow I will start some experimenting with redundant vpc >> > >> routers. >> > >>>>>>>>>>>> This is to check up on any findings and requirements that >> you >> > >> might >> > >>>>>>>>>>>> have on this. Once again I would not like to waste work on >> > this >> > >> as it >> > >>>>>>>>>>>> is really a globally usable feature that is probably >> > universal. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> please let me know your status on this. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> If any of you are coming to the cloudstack day in London >> next >> > >> week, >> > >>>>>>>>>>>> let's meetup next thursday. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> kind regards, >> > >>>>>>>>>>>> Daan Hoogland >> > >>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> -- >> > >>>>>>>>>>> Karl O. Harris >> > >>>>>>>>>>> Cloud Software Engineer >> > >>>>>>>>>>> Sungard Availability Services >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> -- >> > >>>>>>>>>> Karl O. Harris >> > >>>>>>>>>> Cloud Software Engineer >> > >>>>>>>>>> Sungard Availability Services >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> -- >> > >>>>> Karl O. Harris >> > >>>>> Cloud Software Engineer >> > >>>>> Sungard Availability Services >> > >>>> >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Karl O. Harris >> > >>> Cloud Software Engineer >> > >>> Sungard Availability Services >> > >> >> > >> >> > >> >> > > >> > > >> > > -- >> > > Karl O. Harris >> > > Cloud Software Engineer >> > > Sungard Availability Services >> > >> > >> >> >> -- >> Karl O. Harris >> Cloud Software Engineer >> Sungard Availability Services >> -- Daan
