I don't think this is off-topic at all. The flip side of ingesting a blueprint is how to create it. Exporting a project is very interesting idea.
--Alex > -----Original Message----- > From: Kelcey Damage (BT) [mailto:kel...@backbonetechnology.com] On > Behalf Of kdam...@apache.org > Sent: Thursday, January 17, 2013 10:24 AM > To: cloudstack-dev@incubator.apache.org > Subject: RE: [DISCUSS] PaaS Enablement: Composite Application Blueprints > (#576) > > My comment about the projects was that as they stand, they are no > recoverable. When you start a project you must expect it to terminate. There > is no easy method to take the work you accomplish in a project and move it > to a production system. > > It would be nice to be able to pack up a whole project in a vApp (Or at > least and 'administrative migration' option, to migrate VM through admin > boundaries) > > But I digress as this discussion is not about the projects themselves. > > -kd > > > >-----Original Message----- > >From: Jie Feng [mailto:jie.f...@citrix.com] > >Sent: Thursday, January 17, 2013 9:47 AM > >To: cloudstack-dev@incubator.apache.org > >Subject: RE: [DISCUSS] PaaS Enablement: Composite Application Blueprints > >(#576) > > > >Great analysis Chris! > > > >In terms of the blueprint model, I am also in favor of something simple. > >CloudFormation can do what vApp is doing, and more. We only need to > >implement a subset as a starter. For example, allow users to describe > multiple > >VMs, template ID for the VMs, and default service offering/disk offering, > >networking for the interested zones, and startup order. Agree with Kelcey > >that it does not need to be self-contained. We can support > >importing/exporting of the blueprint to vApp (OVA/OVF) which contains > both > >the OVF descriptor and the template image files as an add-on feature. > > > >Do we need to map blueprints to project? I thought project is more like a > >shared workspace. IMO, we should have the flexibility to deploy multiple > >blueprints and any VMs to a project. > > > >Jie > > > >> -----Original Message----- > >> From: Kelceydamage@bbits [mailto:kel...@bbits.ca] > >> Sent: Thursday, January 17, 2013 1:51 AM > >> To: cloudstack-dev@incubator.apache.org > >> Subject: Re: [DISCUSS] PaaS Enablement: Composite Application > >> Blueprints > >> (#576) > >> > >> This was a good read, I think the simpler vApp method makes sense for > >> an infrastructure tool. And could be a great opportunity to rework > >> projects into a more useful system. > >> > >> I would not want to make the blueprints too complex, but my main issue > >> with vApp methodology is how a vApp is completely self-contained. They > >> eat up storage like mad, when they should really just be a collection > >> of existing provisioning resources and templates. > >> > >> Example: > >> > >> A vApp when created copies the included VM temples to create a larger > >> template. It would be better for our blueprints to simply point to the > >> included templates like 'symlinks'. Providing almost zero increase to > >> storage(only meta-data) and making the feature seriously competitive. > >> > >> The main logistics issue would be ensuring a blueprint health check > >> incase someone deletes the linked templates underneath. Or maybe a > >> dependancy error when someone tries to remove a template that's > linked > >> to one or more blueprints. This error could inform the admin that the > >> template is in use by a blueprint that must be deleted first. > >> > >> Thanks > >> > >> Sent from my iPhone > >> > >> On Jan 16, 2013, at 11:51 PM, Chris Sears <chris.x.se...@sungard.com> > >> wrote: > >> > >> > After reading up a bit more on AWS CloudFormation [1] and > >> > OpenStack's work-alike Heat [2], they actually differ quite a bit > >> > from VMware vApps. I thought it might be worth talking about some of > >the differences. > >> > > >> > CloudFormation is a general purpose provisioning system for AWS > >> > services, but under the hood, it's essentially a workflow engine > >> > (similar to vCenter Orchestrator or HP Operations Orchestrator). The > >> > problem CloudFormation is solving is runbook automation, where you > >> > might have a 20 step process for deploying an app into AWS that you > >> > want to completely automate. It competes with tools like Puppet and > >> > Chef > >> (although they can work together). > >> > > >> > The templates used by CloudFormation are less config files and more > >> > like scripts or XSLT. They include variables, user inputs, outputs, > >> > flow control, and a small standard library of functions. [3] > >> > Templates require testing and debugging. It's possible to have > >> > run-time bugs related to a wait condition that cause a deadlock and > trigger > >a rollback. > >> > > >> > When you execute a CloudFormation template, the resulting set of > >> > provisioned resources is called a "stack". A stack could be a > >> > multi-tier app with several VMs, but it could also be 3 empty VPCs, > >> > an SNS queue, and an IAM user. If we follow a similar convention, a > >> > stack would not necessarily map onto a project. A stack might > >> > include VMs in multiple projects. Or no VMs at all. > >> > > >> > To extend the workflow coordination and configuration capabilities > >> > into the guest OS, CloudFormation provides a set of helper scripts > >> > that can fill a similar roll as cloud-init, creating files and > >> > running commands with content and arguments dynamically generated > >> > from the template. [4] > >> > > >> > It helped me to look at some real templates to understand everything > >> > that CloudFormation was doing. Here are a couple examples... (the > >> > first provisions an Active Directory server, the second creates an > >> > instance of > >> > OpenShift) > >> > https://s3.amazonaws.com/cloudformation-templates-us-east- > >> 1/Windows_Si > >> > ngle_Server_Active_Directory.template > >> > > >> > >https://github.com/openstack/heat/blob/master/templates/OpenShift.te > m > >> p > >> > late > >> > > >> > By contrast, vApps are much simpler. They are effectively just a > >> > collection of one or more VMs, their associated network settings and > >> > some limited additional metadata. You can start and stop a vApp. You > >> > can import or export a vApp as an OVF file if you wanted to move it > >> > from one cloud to another. A vApp could roughly map to a project. > >> > > >> > To me, it sounds like we need vApps, with some limited features of > >> > CloudFormation. Just enough to enable a user experience of clicking > >> > a "Launch Blueprint" button and having everything magically > >> > provisioned, like AWS does here: > >> > http://aws.amazon.com/cloudformation/aws-cloudformation- > templates/ > >> > > >> > Thoughts? > >> > > >> > - Chris > >> > > >> > [1]: http://aws.amazon.com/cloudformation/faqs/ > >> > [2]: https://github.com/openstack/heat > >> > [3]: > >> > > >> > >http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/tem > pl > >> ate > >> > -anatomy.html > >> > [4]: > >> > > >http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn- > >> help > >> > er-scripts-reference.html > >> > > >> > > >> > On Wed, Jan 16, 2013 at 1:59 PM, Mohammad Nour El-Din < > >> > nour.moham...@gmail.com> wrote: > >> > > >> >> Hi > >> >> > >> >> Sent from my Samsung Galaxy S3 > >> >> Apologies for any typos > >> >> On Jan 16, 2013 6:01 PM, "Shane Witbeck" > <sh...@digitalsanctum.com> > >> wrote: > >> >>> > >> >>> +1 for YAML. After all, YAML is a superset of JSON [1] which is > >> >>> +the > >> >> primary objective of the 1.2 spec. > >> >>> > >> >>> Here are my pros and cons: > >> >>> > >> >>> Pros: > >> >>> - relational anchors and aliases (potential here for less > >> >>> verbosity > >> >> through reuse) > >> >>> - more readable > >> >>> - comments > >> >>> - projects which I respect, choose yaml over json for similar > >> >>> types of > >> >> configuration [2], [3] > >> >>> > >> >>> Cons: > >> >>> - more complex to parse, less mature libraries compared to json > >> >>> > >> >>> All this being said, I would advocate that more initial focus > >> >>> should be > >> >> on the "model" and how it solves the problem of defining the > >"blueprint". > >> >> Serialization format discussion should be secondary and around > >> >> whatever best fits the model, not the other way around. > >> >> > >> >> +1 on that > >> >> > >> >>> > >> >>> > >> >>> [1] http://yaml.org/spec/1.2/spec.html#id2759572 > >> >>> [2] > >> >> > http://www.elasticsearch.org/guide/reference/setup/configuration.ht > >> >> ml > >> >>> [3] > >> >> https://github.com/cloudfoundry/bosh- > >> release/blob/master/micro/aws.ym > >> >> l > >> >>> > >> >>> > >> >>> -- > >> >>> Shane Witbeck > >> >>> > >> >>> > >> >>> On Thursday, January 10, 2013 at 1:20 PM, Kelcey Damage (BT) wrote: > >> >>> > >> >>>> +1 for YAML, I find JSON good for when I never look at the data > >> >>>> +and > >> >> simply pass to a Python dictionary, however for interaction I am > >> >> going to back the YAML choice. > >> >>>> > >> >>>> KELCEY DAMAGE > >> >>>> Infrastructure Systems Architect > >> >>>> www.backbonetechnology.com > >> (http://www.backbonetechnology.com) > >> >>>> > >> >> ------------------------------------------------------------------- > >> >> -- > >> >> ---- > >> >>>> kel...@backbonetechnology.com > >> >>>> (mailto:kel...@backbonetechnology.com) > >> >>>> > >> >>>> address: 55 East 7th Ave, Vancouver, BC, V5T 1M4 > >> >>>> tel: +1 604 713 8560 ext:114 > >> >>>> fax: +1 604 605 0964 > >> >>>> skype: kelcey.damage > >> >>>> > >> >>>> > >> >>>> > >> >>>>> -----Original Message----- > >> >>>>> From: Alex Heneveld [mailto:alex.henev...@cloudsoftcorp.com] > >> >>>>> Sent: Wednesday, January 09, 2013 4:39 PM > >> >>>>> To: cloudstack-dev@incubator.apache.org (mailto: > >> >> cloudstack-dev@incubator.apache.org) > >> >>>>> Cc: Min Chen > >> >>>>> Subject: Re: [DISCUSS] PaaS Enablement: Composite Application > >> >> Blueprints > >> >>>>> (#576) > >> >>>>> > >> >>>>> > >> >>>>> Hi Min, Jie, > >> >>>>> > >> >>>>> Min, nice questions. I've given my answers in-line. > >> >>>>> > >> >>>>> Jie, my answers also respond to some of your points, re TOSCA > >> >>>>> and YAML/JSON. > >> >>>>> > >> >>>>> > >> >>>>> On 08/01/2013 17:50, Min Chen wrote: > >> >>>>>> +1 on this feature to extend CloudStack from pure IaaS to PaaS > >> >>>>>> +or > >> >> SaaS > >> >>>>>> area. Some questions in my mind: > >> >>>>>> > >> >>>>>> 1) Does your proposed blueprint only contain functional > >> >>>>>> component description or contain both functional and > deployment > >aspects? > >> >>>>>> For example, for a 3-tier app containing three functional > >> >>>>>> components > >> >> like > >> >>>>>> web tier, server, and DB, users can define different deployment > >> >> models > >> >>>>>> for the same functional components, like small deployment > >> >> (co-hosting > >> >>>>>> all components on one VM), or medium or large deployment. > >> >>>>>> > >> >>>>> > >> >>>>> Great use case. Introduces a _lot_ of complexity: > >> >>>>> > >> >>>>> (1) allow parameters to be passed to the blueprint (e.g. "size") > >> >>>>> (2) separate the components that are to be created from the > >> >> customizations > >> >>>>> performed > >> >>>>> (so you could define customizations for webserver, appserver, > >> >>>>> and > >> >> data, > >> >>>>> then either > >> >>>>> apply the customizations either to a single VM (small), distinct > >> >>>>> VM's (medium), or distinct VM groups (large)) > >> >>>>> (3) allow blueprints to refer to other blueprints (so the > >> >>>>> "webserver customization" lives as its own reusable module) > >> >>>>> (4) allow conditional branching based on parameters (e.g. if > >> >>>>> user selects "size=small" then run all customisations on > >> >> single VM) > >> >>>>> > >> >>>>> I'd be inclined to DEFER them as features, until phase 2 or 3, > >> >>>>> but we > >> >> should > >> >>>>> consider how the description might support them as I'd like to > >> >>>>> see > >> >> some of > >> >>>>> this functionality eventually. > >> >>>>> > >> >>>>> What do other people think? This is obvious functionality to > >> >>>>> want, > >> >> what's the > >> >>>>> right way to support it but without making the description > >> >>>>> language complicated? > >> >>>>> > >> >>>>> (BTW one of the reasons I think TOSCA could be a good choice is > >> >>>>> that > >> >> it has > >> >>>>> considered many of these. You can define properties (1 above), > >> >> arbitrary > >> >>>>> nodes (2), and references to other definitions (3). It also has > >> >>>>> the > >> >> concepts of > >> >>>>> requirements / capabilities, relationships, policies, and plans > >> >>>>> -- > >> >> some of which > >> >>>>> could be used to support (4) > >> >>>>> eventually.) > >> >>>>> > >> >>>>>> 2) Have you thought of creating Service offerings from your > >> >>>>>> defined blueprints? This may allow CloudStack to provide some > >> >>>>>> SaaS > >> >> functionalities. > >> >>>>>> > >> >>>>> > >> >>>>> I like this idea, although it might break assumptions made > >> >>>>> elsewhere > >> >> about > >> >>>>> service offerings so we should look at it carefully. > >> >>>>> > >> >>>>>> 3) As for blueprint description language, easy-to-read and > >> >>>>>> easy-to-edit by human being should be necessary if we don't > >> >>>>>> provide any visual GUI tools to create/edit a blueprint. I have > >> >>>>>> seen some > >> >> JSON > >> >>>>>> format before in BMC Cloud LifeCycle product 2.0. > >> >>>>>> > >> >>>>> > >> >>>>> Wow JSON seems popular so far. And yet YAML is more concise > and > >> >>>>> expressive. Designed to be easy for people to read and write > >> >> configuration > >> >>>>> just like this. Whereas JSON is designed for serializing > >> >>>>> objects, in > >> >> a way that > >> >>>>> isn't too hard for people. YAML seems the right language for > >> >>>>> this > >> >> purpose to > >> >>>>> me -- and my experience has been that it is easier to read and > >> >>>>> to > >> >> write, > >> >>>>> without the litter of curly braces and quotes. > >> >>>>> > >> >>>>> But I don't want to be a language bigot! I can go with JSON if > >> >>>>> that's > >> >> what > >> >>>>> people want. :) > >> >>>>> > >> >>>>>> 4) For a multi-tier application, blueprint should not only > >> >>>>>> describe different components, but also connections among > those > >> >>>>>> components, like open port, LB and FW rules among them. I > >> >>>>>> assume that your blueprint is also taking that into > >> >>>>>> consideration, and your backend orchestration layer can provide > >> >>>>>> enough flexibility to provision any such app, seems not an easy > >> >>>>>> task to me since I am > >> new to CS. > >> >>>>>> > >> >>>>> > >> >>>>> Yeah, this is where it gets interesting. A common trick I've > >> >>>>> seen is > >> >> to use > >> >>>>> hostnames for a lot of the connection configuration, and have a > >> >>>>> few > >> >> hard- > >> >>>>> coded patterns in there (e.g. for groups, load-balancers). But > >> >>>>> that > >> >> only gets > >> >>>>> you so far, and then you hit a wall (or spin up a different > >> >> orchestrator). > >> >>>>> > >> >>>>> Using requirements/capabilities and typed relationships gives a > >> >>>>> much > >> >> more an > >> >>>>> elegant solution. (Not necessarily TOSCA, but these concepts are > >> >>>>> well developed there!) > >> >>>>> > >> >>>>> In any case let's make sure the use cases include interesting > >> >> situations like > >> >>>>> this, and we can compare approaches. This absolutely DOES need > >> >>>>> to be > >> >> in > >> >>>>> phase 1 I think. > >> >>>>> > >> >>>>>> 5) From a normal user perspective, I would really love to see a > >> >>>>>> GUI tool developed around this to allow user to visually > >> >>>>>> create/edit a multi-tier app blueprint instead of using a > TextEditor. > >> >>>>>> > >> >>>>> > >> >>>>> Nice to hear. Me too. The description, API, and backend come > >> >>>>> first, > >> >> with an > >> >>>>> initial set of features, but I'd love to see the GUI in phase 2. > >> >>>>> > >> >>>>>> Thanks > >> >>>>>> -min > >> >>>>>> > >> >>>>> > >> >>>>> Keep it coming please. > >> >>>>> > >> >>>>> --Alex > >> >>> > >> >>
