I have updated the FS with the addition of the DeploymentPlanningManager entity.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups -Prachi -----Original Message----- From: Prachi Damle Sent: Thursday, February 07, 2013 1:38 PM To: Alex Huang; cloudstack-dev@incubator.apache.org Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules Alex, Thanks for the detailed review. I will couple the affinity design with the deployment planner refactoring I had next in line then. Will update the FS with these details. -Prachi -----Original Message----- From: Alex Huang Sent: Wednesday, February 06, 2013 12:00 PM To: Prachi Damle; cloudstack-dev@incubator.apache.org Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules Prachi, If it's an elective option, then it shouldn't have options in the deployvm api. We can't decouple the code but then say it's coupled at deployvm api call. I think it makes sense to have the affinity be part of orchestration. If so, then it makes sense to do this. - Write DeploymentPlanningManager which contains planning for volume, vm, and network. - For VM deployment, DPM goes through the following stages - Affinity - to set deploymentplan scope and exclude list (we might think about merging the two). - DeploymentPlanner - to use heuristics to find the best spot to place the vm/volume. - Allocator - which matches the requirements to capabilities of the physical resource. If you think about it, then it makes sense because each matches to the functionality it serves. - Affinity - matches user preference to narrow down where to deploy. - DeploymentPlanner - matches algorithms set by Administrator to give best performance/value/feature - Allocator - matches physical requirements. This would mean taking Allocator out of DeploymentPlanner implementations. It is very similar to your intent with an AffinityDeploymentPlanner. I think actually this works out very well with the idea of reservations as well. You might think about doing the two together as a refactor of Deployment Planning. --Alex > -----Original Message----- > From: Prachi Damle > Sent: Tuesday, February 05, 2013 11:39 PM > To: Alex Huang; cloudstack-dev@incubator.apache.org > Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules > > Hi Alex, > > Thanks for the review, I have answered inline. Please comment. > I guess the FS needs more description reasoning the 2-component design > to avoid confusion. > > -Prachi > > -----Original Message----- > From: Alex Huang > Sent: Tuesday, February 05, 2013 4:49 PM > To: Prachi Damle; cloudstack-dev@incubator.apache.org > Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules > > Hi Prachi, > > A few comments about this spec. > > - What is the error when planning fails? What details will it give? > > > [Prachi] I was planning to still rely on the deployment planner to > throw exception since planner will be the component that will find out > that placement is not possible. > But it is a good point and I will add a custom exception that the > AffinityProcessor can throw if upon analysis it finds the affinity > rule cannot be followed. This will save the planner execution. > > > - Why are HostAffinityProcessor and HostAntiAffinityProcessor > different implementations? The requirements says they should be > specified concurrently so shouldn't they be the same processor? Or > are you planning for this adapter to be an adapter chain? > > > [Prachi] Yes I am planning this to be a chain of adapters, each can > handle a specific affinity type. > > - I'm not really sure I understand the design. It would seem like > this can be designed in two ways. > 1. Affinity Group is an integrated part of cloud-engine and > AffinityGroupProcessor is ran before all DeploymentPlanners to set the > scope of DeploymentPlan and ExcludeList. If this is the case, I don't > understand why there is a AffinityGroupPlanner. > 2. The entire thing is implemented as a DeploymentPlanner. If it is > implemented as a DeploymentPlanner, then why do we need the > AffinityGroupProcessor interface? Why not just have it as a property > of that DeploymentPlanner implementation? > After giving it some thought, I think it is important that affinity > group is part of core orchestration. If so then I would prefer 1. > The current spec has both so I'm confused on this. > - The getType() of the AffinityGroupProcessor seems like it's more > similar to the getName of the adapter already, which needs to be > unique. I think you just need more descriptions on what the processor > does so the end user can understand how to use it? > > [Prachi] Since use of the affinity groups will be done during VM > placement, I wanted to stick to add all logic to DeploymentPlanner. > This will make sure that anytime the planner gets called (deployment, > restart, migration, HA) affinity groups are take care of. Still I see > the need of being able to plugin custom implementations for various > types of affinity/anti-affinity. So pushed out the handling of the > affinityType to a separate adapter. > > Thus AffinityPlanner could be as simple as just running through the > chain of AffinityGroupProcessor's and it will not change even if we > introduce any new affinity types. It will be the hook in Cloudstack, > for the plugins implementing affinity types > > As you say we could keep this part of cloud-engine and run through the > chain of AffinityGroupProcessor before planners - but it will not take > care of other usecases that directly call planners for placement. > > - Can you check with Min on her creation of the REST API? Can we > start using the REST only API from here on? > [Prachi ]Sure will sync up with Min on REST API. > > --Alex > > > -----Original Message----- > > From: Prachi Damle [mailto:prachi.da...@citrix.com] > > Sent: Tuesday, February 05, 2013 2:42 PM > > To: cloudstack-dev@incubator.apache.org > > Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules > > > > Hey all, > > > > Following up on this feature discussion. I have put up an initial > > draft of the FS for this feature here: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity > > - > > Anti-affinity+groups > > > > As per discussions below, the scope of the feature now consists of a > > generic framework for defining affinity groups in CloudStack and a > > default implementation to support host affinity and anti-affinity. > > > > Please provide your comments. > > > > Thanks, > > Prachi > > > > -----Original Message----- > > From: Chip Childers [mailto:chip.child...@sungard.com] > > Sent: Monday, January 07, 2013 6:30 AM > > To: cloudstack-dev@incubator.apache.org > > Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules > > > > On Mon, Jan 7, 2013 at 1:22 AM, Chris Sears > > <chris.x.se...@sungard.com> > > wrote: > > > Greetings, > > > > > > I understand the motivation for a feature like this, but I'm > > > concerned that the concepts of affinity and anti-affinity might > > > not be appropriate cloud-level abstractions to expose to end users. > > > Also, it might be difficult to effectively automate decisions > > > about fault and performance domains, both of which would vary > > > greatly between > > deployments. > > > > > > Using anti-affinity rules to make sure HA-related VMs aren't > > > placed on the same host seems like the most critical use case. > > > What if we narrowed the scope of the feature to just address that issue? > > > Building on Chiradeep's idea, VMs could have an anti-affinity > > > group > attribute. > > > VMs in the same anti-affinity group must be placed on different hosts. > > > For the first implementation, the guarantee would only apply to > > > initial > > provisioning. > > > > It could actually apply any time a planner is used to select a host > > (which I think also includes CloudStack "HA"). > > > > > @Manan, would that be sufficient? > > > > > > Regards > > > > > > - Chris > > > > > > > > > On Fri, Jan 4, 2013 at 6:50 PM, Prachi Damle > > <prachi.da...@citrix.com>wrote: > > > > > >> Yes, requirements seem vague. What parameters define > > >> affinity/anti-affinity? > > >> > > >> Requirements mention > > >> >> For each VM, users should be able to provide both (Affinity > > >> >> VMs and > > >> Anti-affinity VMs) lists concurrently. For example, VM-A can have > > >> affinity with VMs B & C and anti-affinity with VMs D & E at the > > >> same > time. > > >> >> When configuring Affinity / anti-affinity for a VM, users > > >> >> should be > > >> allowed to provide a list of affinity / anti-affinity VMs (via > > >> API) or select affinity /anti-affinity VMs from a list (via UI) > > >> > > >> When user specifies VM-A can have affinity with VMs B & C does > > >> that mean they should be placed on same pod or same > > >> hypervisor(cluster or > > >> host) by the allocation logic? > > >> > > >> -Prachi > > >> -----Original Message----- > > >> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] > > >> Sent: Thursday, January 03, 2013 6:06 PM > > >> To: CloudStack DeveloperList > > >> Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules > > >> > > >> Actually the proposal is quite vague. > > >> What does affinity mean to the end-user? > > >> What guarantees are being asked for? > > >> - the vms are on the same hypervisor (affinity) > > >> - the vms are not on the same hypervisor (anti) > > >> - the vms are interconnected by a high-speed interconnect > > >> (affinity) > > >> - the vms are in different failure domains (host/cluster/pod) > > >> > > >> I find the concept of affinity groups useful. > > >> A possible workflow would be > > >> 1. Create an affinity group of type 'Foo' > > >> 1a. Group type indicates the guarantee 2. Create a VM in the > > >> group > > >> > > >> VMs can only leave groups on vm destruction. > > >> > > >> But without the specific type of guarantee, it is hard to discuss > > >> this proposal. > > >> > > >> On 1/3/13 4:23 PM, "Manan Shah" <manan.s...@citrix.com> wrote: > > >> > > >> >Hi, > > >> > > > >> >I would like to propose a new feature for enabling Affinity / > > >> >Anti-affinity rules in CS 4.1. I have created a JIRA ticket and > > >> >provided the requirements at the following location. Please > > >> >provide feedback on the requirements. > > >> > > > >> >JIRA Ticket: > > >> >https://issues.apache.org/jira/browse/CLOUDSTACK-739 > > >> >Requirements: > > >> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Affinity+ > > >> >- > > +An > > >> >ti- > > >> >aff > > >> >i > > >> >nity+rules > > >> > > > >> > > > >> >Regards, > > >> >Manan Shah > > >> > > > >> > > >> > > >>
