Great analysis Chris! In terms of the blueprint model, I am also in favor of something simple. CloudFormation can do what vApp is doing, and more. We only need to implement a subset as a starter. For example, allow users to describe multiple VMs, template ID for the VMs, and default service offering/disk offering, networking for the interested zones, and startup order. Agree with Kelcey that it does not need to be self-contained. We can support importing/exporting of the blueprint to vApp (OVA/OVF) which contains both the OVF descriptor and the template image files as an add-on feature.
Do we need to map blueprints to project? I thought project is more like a shared workspace. IMO, we should have the flexibility to deploy multiple blueprints and any VMs to a project. Jie > -----Original Message----- > From: Kelceydamage@bbits [mailto:kel...@bbits.ca] > Sent: Thursday, January 17, 2013 1:51 AM > To: cloudstack-dev@incubator.apache.org > Subject: Re: [DISCUSS] PaaS Enablement: Composite Application Blueprints > (#576) > > This was a good read, I think the simpler vApp method makes sense for an > infrastructure tool. And could be a great opportunity to rework projects into > a more useful system. > > I would not want to make the blueprints too complex, but my main issue > with vApp methodology is how a vApp is completely self-contained. They eat > up storage like mad, when they should really just be a collection of existing > provisioning resources and templates. > > Example: > > A vApp when created copies the included VM temples to create a larger > template. It would be better for our blueprints to simply point to the > included templates like 'symlinks'. Providing almost zero increase to > storage(only meta-data) and making the feature seriously competitive. > > The main logistics issue would be ensuring a blueprint health check incase > someone deletes the linked templates underneath. Or maybe a dependancy > error when someone tries to remove a template that's linked to one or more > blueprints. This error could inform the admin that the template is in use by a > blueprint that must be deleted first. > > Thanks > > Sent from my iPhone > > On Jan 16, 2013, at 11:51 PM, Chris Sears <chris.x.se...@sungard.com> > wrote: > > > After reading up a bit more on AWS CloudFormation [1] and OpenStack's > > work-alike Heat [2], they actually differ quite a bit from VMware > > vApps. I thought it might be worth talking about some of the differences. > > > > CloudFormation is a general purpose provisioning system for AWS > > services, but under the hood, it's essentially a workflow engine > > (similar to vCenter Orchestrator or HP Operations Orchestrator). The > > problem CloudFormation is solving is runbook automation, where you > > might have a 20 step process for deploying an app into AWS that you > > want to completely automate. It competes with tools like Puppet and Chef > (although they can work together). > > > > The templates used by CloudFormation are less config files and more > > like scripts or XSLT. They include variables, user inputs, outputs, > > flow control, and a small standard library of functions. [3] Templates > > require testing and debugging. It's possible to have run-time bugs > > related to a wait condition that cause a deadlock and trigger a rollback. > > > > When you execute a CloudFormation template, the resulting set of > > provisioned resources is called a "stack". A stack could be a > > multi-tier app with several VMs, but it could also be 3 empty VPCs, an > > SNS queue, and an IAM user. If we follow a similar convention, a stack > > would not necessarily map onto a project. A stack might include VMs in > > multiple projects. Or no VMs at all. > > > > To extend the workflow coordination and configuration capabilities > > into the guest OS, CloudFormation provides a set of helper scripts > > that can fill a similar roll as cloud-init, creating files and running > > commands with content and arguments dynamically generated from the > > template. [4] > > > > It helped me to look at some real templates to understand everything > > that CloudFormation was doing. Here are a couple examples... (the > > first provisions an Active Directory server, the second creates an > > instance of > > OpenShift) > > https://s3.amazonaws.com/cloudformation-templates-us-east- > 1/Windows_Si > > ngle_Server_Active_Directory.template > > > https://github.com/openstack/heat/blob/master/templates/OpenShift.tem > p > > late > > > > By contrast, vApps are much simpler. They are effectively just a > > collection of one or more VMs, their associated network settings and > > some limited additional metadata. You can start and stop a vApp. You > > can import or export a vApp as an OVF file if you wanted to move it > > from one cloud to another. A vApp could roughly map to a project. > > > > To me, it sounds like we need vApps, with some limited features of > > CloudFormation. Just enough to enable a user experience of clicking a > > "Launch Blueprint" button and having everything magically provisioned, > > like AWS does here: > > http://aws.amazon.com/cloudformation/aws-cloudformation-templates/ > > > > Thoughts? > > > > - Chris > > > > [1]: http://aws.amazon.com/cloudformation/faqs/ > > [2]: https://github.com/openstack/heat > > [3]: > > > http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/templ > ate > > -anatomy.html > > [4]: > > http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn- > help > > er-scripts-reference.html > > > > > > On Wed, Jan 16, 2013 at 1:59 PM, Mohammad Nour El-Din < > > nour.moham...@gmail.com> wrote: > > > >> Hi > >> > >> Sent from my Samsung Galaxy S3 > >> Apologies for any typos > >> On Jan 16, 2013 6:01 PM, "Shane Witbeck" <sh...@digitalsanctum.com> > wrote: > >>> > >>> +1 for YAML. After all, YAML is a superset of JSON [1] which is the > >> primary objective of the 1.2 spec. > >>> > >>> Here are my pros and cons: > >>> > >>> Pros: > >>> - relational anchors and aliases (potential here for less verbosity > >> through reuse) > >>> - more readable > >>> - comments > >>> - projects which I respect, choose yaml over json for similar types > >>> of > >> configuration [2], [3] > >>> > >>> Cons: > >>> - more complex to parse, less mature libraries compared to json > >>> > >>> All this being said, I would advocate that more initial focus should > >>> be > >> on the "model" and how it solves the problem of defining the "blueprint". > >> Serialization format discussion should be secondary and around > >> whatever best fits the model, not the other way around. > >> > >> +1 on that > >> > >>> > >>> > >>> [1] http://yaml.org/spec/1.2/spec.html#id2759572 > >>> [2] > >> http://www.elasticsearch.org/guide/reference/setup/configuration.html > >>> [3] > >> https://github.com/cloudfoundry/bosh- > release/blob/master/micro/aws.ym > >> l > >>> > >>> > >>> -- > >>> Shane Witbeck > >>> > >>> > >>> On Thursday, January 10, 2013 at 1:20 PM, Kelcey Damage (BT) wrote: > >>> > >>>> +1 for YAML, I find JSON good for when I never look at the data and > >> simply pass to a Python dictionary, however for interaction I am > >> going to back the YAML choice. > >>>> > >>>> KELCEY DAMAGE > >>>> Infrastructure Systems Architect > >>>> www.backbonetechnology.com > (http://www.backbonetechnology.com) > >>>> > >> --------------------------------------------------------------------- > >> ---- > >>>> kel...@backbonetechnology.com > >>>> (mailto:kel...@backbonetechnology.com) > >>>> > >>>> address: 55 East 7th Ave, Vancouver, BC, V5T 1M4 > >>>> tel: +1 604 713 8560 ext:114 > >>>> fax: +1 604 605 0964 > >>>> skype: kelcey.damage > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Alex Heneveld [mailto:alex.henev...@cloudsoftcorp.com] > >>>>> Sent: Wednesday, January 09, 2013 4:39 PM > >>>>> To: cloudstack-dev@incubator.apache.org (mailto: > >> cloudstack-dev@incubator.apache.org) > >>>>> Cc: Min Chen > >>>>> Subject: Re: [DISCUSS] PaaS Enablement: Composite Application > >> Blueprints > >>>>> (#576) > >>>>> > >>>>> > >>>>> Hi Min, Jie, > >>>>> > >>>>> Min, nice questions. I've given my answers in-line. > >>>>> > >>>>> Jie, my answers also respond to some of your points, re TOSCA and > >>>>> YAML/JSON. > >>>>> > >>>>> > >>>>> On 08/01/2013 17:50, Min Chen wrote: > >>>>>> +1 on this feature to extend CloudStack from pure IaaS to PaaS or > >> SaaS > >>>>>> area. Some questions in my mind: > >>>>>> > >>>>>> 1) Does your proposed blueprint only contain functional component > >>>>>> description or contain both functional and deployment aspects? > >>>>>> For example, for a 3-tier app containing three functional > >>>>>> components > >> like > >>>>>> web tier, server, and DB, users can define different deployment > >> models > >>>>>> for the same functional components, like small deployment > >> (co-hosting > >>>>>> all components on one VM), or medium or large deployment. > >>>>>> > >>>>> > >>>>> Great use case. Introduces a _lot_ of complexity: > >>>>> > >>>>> (1) allow parameters to be passed to the blueprint (e.g. "size") > >>>>> (2) separate the components that are to be created from the > >> customizations > >>>>> performed > >>>>> (so you could define customizations for webserver, appserver, and > >> data, > >>>>> then either > >>>>> apply the customizations either to a single VM (small), distinct > >>>>> VM's (medium), or distinct VM groups (large)) > >>>>> (3) allow blueprints to refer to other blueprints (so the > >>>>> "webserver customization" lives as its own reusable module) > >>>>> (4) allow conditional branching based on parameters (e.g. if user > >>>>> selects "size=small" then run all customisations on > >> single VM) > >>>>> > >>>>> I'd be inclined to DEFER them as features, until phase 2 or 3, but > >>>>> we > >> should > >>>>> consider how the description might support them as I'd like to see > >> some of > >>>>> this functionality eventually. > >>>>> > >>>>> What do other people think? This is obvious functionality to want, > >> what's the > >>>>> right way to support it but without making the description > >>>>> language complicated? > >>>>> > >>>>> (BTW one of the reasons I think TOSCA could be a good choice is > >>>>> that > >> it has > >>>>> considered many of these. You can define properties (1 above), > >> arbitrary > >>>>> nodes (2), and references to other definitions (3). It also has > >>>>> the > >> concepts of > >>>>> requirements / capabilities, relationships, policies, and plans -- > >> some of which > >>>>> could be used to support (4) > >>>>> eventually.) > >>>>> > >>>>>> 2) Have you thought of creating Service offerings from your > >>>>>> defined blueprints? This may allow CloudStack to provide some > >>>>>> SaaS > >> functionalities. > >>>>>> > >>>>> > >>>>> I like this idea, although it might break assumptions made > >>>>> elsewhere > >> about > >>>>> service offerings so we should look at it carefully. > >>>>> > >>>>>> 3) As for blueprint description language, easy-to-read and > >>>>>> easy-to-edit by human being should be necessary if we don't > >>>>>> provide any visual GUI tools to create/edit a blueprint. I have > >>>>>> seen some > >> JSON > >>>>>> format before in BMC Cloud LifeCycle product 2.0. > >>>>>> > >>>>> > >>>>> Wow JSON seems popular so far. And yet YAML is more concise and > >>>>> expressive. Designed to be easy for people to read and write > >> configuration > >>>>> just like this. Whereas JSON is designed for serializing objects, > >>>>> in > >> a way that > >>>>> isn't too hard for people. YAML seems the right language for this > >> purpose to > >>>>> me -- and my experience has been that it is easier to read and to > >> write, > >>>>> without the litter of curly braces and quotes. > >>>>> > >>>>> But I don't want to be a language bigot! I can go with JSON if > >>>>> that's > >> what > >>>>> people want. :) > >>>>> > >>>>>> 4) For a multi-tier application, blueprint should not only > >>>>>> describe different components, but also connections among those > >>>>>> components, like open port, LB and FW rules among them. I assume > >>>>>> that your blueprint is also taking that into consideration, and > >>>>>> your backend orchestration layer can provide enough flexibility > >>>>>> to provision any such app, seems not an easy task to me since I am > new to CS. > >>>>>> > >>>>> > >>>>> Yeah, this is where it gets interesting. A common trick I've seen > >>>>> is > >> to use > >>>>> hostnames for a lot of the connection configuration, and have a > >>>>> few > >> hard- > >>>>> coded patterns in there (e.g. for groups, load-balancers). But > >>>>> that > >> only gets > >>>>> you so far, and then you hit a wall (or spin up a different > >> orchestrator). > >>>>> > >>>>> Using requirements/capabilities and typed relationships gives a > >>>>> much > >> more an > >>>>> elegant solution. (Not necessarily TOSCA, but these concepts are > >>>>> well developed there!) > >>>>> > >>>>> In any case let's make sure the use cases include interesting > >> situations like > >>>>> this, and we can compare approaches. This absolutely DOES need to > >>>>> be > >> in > >>>>> phase 1 I think. > >>>>> > >>>>>> 5) From a normal user perspective, I would really love to see a > >>>>>> GUI tool developed around this to allow user to visually > >>>>>> create/edit a multi-tier app blueprint instead of using a TextEditor. > >>>>>> > >>>>> > >>>>> Nice to hear. Me too. The description, API, and backend come > >>>>> first, > >> with an > >>>>> initial set of features, but I'd love to see the GUI in phase 2. > >>>>> > >>>>>> Thanks > >>>>>> -min > >>>>>> > >>>>> > >>>>> Keep it coming please. > >>>>> > >>>>> --Alex > >>> > >>
