I am able to create singel instance LAMP-stack with Chiradeeps's implementation. I will spend a little time, understanding his code (new to ruby). Will take it forward from there.
Thanks, Dharmesh On Tue, May 7, 2013 at 1:16 PM, Sebastien Goasguen <run...@gmail.com> wrote: > > On May 1, 2013, at 7:27 AM, Dharmesh Kakadia < > dharmesh.kaka...@research.iiit.ac.in> wrote: > > > Sebastien and Chiradeep, thanks for the comments !! That clarified a lot > of > > things. I just read Chiradeep's blog ( > > > http://cloudierthanthou.wordpress.com/2013/04/26/stackmate-execute-cloudformation-templates-on-cloudstack/ > ) > > which details the service. > > > > I am proposing a server side implementation of cloudformation. > > > > I misunderstood the ReST and Query API. Thanks for correcting. > Information > > here(http://gehrcke.de/2009/06/aws-about-api/) helped me. In case we > want > > to use existing AWS tools for cloudformation, we also would be designing > > Query API, not ReST. > > > > Sorry for the confusion regarding cloudmonkey. I was proposing to > integrate > > cloudformation API into cloudstack source code, directly and add > > corresponding support in cloudmonkey. But as you suggested, it might be > > easy to start with prototype decoupled from cloudstack (Uses cloudstack > API > > and does not reside in cloudstack). I assume by existing cloudformation > > tools you mean AWS tools( > > http://aws.amazon.com/developertools/AWS-CloudFormation). Reusing them > will > > be a really good idea. > > Yes first step should be with cloud formation (stackmate) outside > cloudstack. And I strongly suggest that you make sure it is compatible with > the AWS tools. > > > > > There are lot of options for configuration mgmt tools. I have used knife > > previously and good to know that it has cloudstack plugin based on fog ( > > https://github.com/fifthecho/knife-cloudstack-fog). Reasons rundeck > looked > > better was support for rollbacking and is full workflow execution engine. > > Finally rundeck can use chef/puppet. I have seen provisonr/whirr and they > > look promising. Definitely a lot to explore here !! > > Yes, I have nothing against rundeck. But it would be one more dependency. > > > > > Thanks for suggesting clear proposal. > > > > Thanks, > > Dharmesh > > > > > > > > > > > > On Wed, May 1, 2013 at 3:07 PM, Sebastien Goasguen <run...@gmail.com> > wrote: > > > >> > >> On Apr 30, 2013, at 4:59 PM, Chiradeep Vittal < > chiradeep.vit...@citrix.com> > >> wrote: > >> > >>> > >>> > >>> On 4/30/13 5:01 AM, "Sebastien Goasguen" <run...@gmail.com> wrote: > >>> > >>>> Dharmesh, see in-line > >>>> > >>>> On Apr 30, 2013, at 5:34 AM, Dharmesh Kakadia <dhkaka...@gmail.com> > >> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> I am Dharmesh Kakdia and interested in project "Integration project > to > >>>>> deploy and use Mesos on a CloudStack based cloud" ( > >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1784) > >>>>> > >>>>> I am working on proposal and want to get feedback. Please provide > >>>>> suggestions :) > >>>>> > >>>>> * > >>>>> > >>>>> Abstract: > >>>>> > >>>>> The project aims to bring cloudformation[1] like service to > cloudstack. > >>>>> One > >>>>> of the prime use-case is cluster computing frameworks on cloudstack. > A > >>>>> cloudformation service will give users and administrators of > cloudstack > >>>>> ability to manage and control a set of resources easily. The > >>>>> cloudformation > >>>>> will allow booting and configuring a set of VMs and form a cluster. > >>>>> Simple > >>>>> example would be LAMP stack. More complex clusters such as mesos or > >>>>> hadoop > >>>>> cluster requires a little more advanced configuration. There is > already > >>>>> some work done by Chiradeep Vittal at this front [5] using route and > >>>> > >>>> it's using ruote: http://ruote.rubyforge.org > >>>> > >>>>> sinatra. In this project, I will implement cloudformation service and > >>>>> demonstrate how to run mesos cluster using it. > >>>> > >>>> You will create cloud formation templates that describe a mesos > cluster > >>>> > >>>>> > >>>>> Mesos: > >>>>> > >>>>> Mesos is a resource management platform for clusters [2]. It aims to > >>>>> increase resource utilization of clusters by sharing cluster > resources > >>>>> among multiple processing frameworks(like MapReduce, MPI, Graph > >>>>> Processing) > >>>>> or multiple instances of same framework. It provides efficient > resource > >>>>> isolation through use of containers. Uses zookeeper for state > >>>>> maintenance > >>>>> and fault tolerance. > >>>>> > >>>>> What can run on mesos ? > >>>>> > >>>>> Spark: A cluster computing framework based on the Resilient > Distributed > >>>>> Datasets (RDDs) abstraction. RDD is more generalized than MapReduce > and > >>>>> can > >>>>> support iterative and interactive computation while retaining fault > >>>>> tolerance, scalability, data locality etc. > >>>>> > >>>>> Hadoop: Hadoop is fault tolerant and scalable distributed computing > >>>>> framework based on MapReduce abstraction. > >>>>> > >>>>> Begel: A graph processing framework based on pregel. > >>>>> > >>>>> and other frameworks like MPI, Hypertable. > >>>>> > >>>>> How to deploy mesos > >>>>> > >>>>> Mesos provides cluster installation scripts [7] for cluster > deployment. > >>>>> There are also scripts available to deploy a cluster on Amazon EC2 > [8]. > >>>> > >>>> It would be nice to see if these scripts can be used as is with the > >>>> CloudStack EC2 service. > >>>> > >>>>> > >>>>> Deliverables: > >>>>> > >>>>> 1. Cloudformation service implementation on cloudstack. > >>>>> > >>>>> 2. Integration of cloudformation with cloudmonkey, CLI tool. > >>>> > >>>> 2. is a little confusing. I believe that what Chiradeep prototype runs > >> on > >>>> the client side. What is needed is a server side implementation. > >>>> That way we could use existing cloudformation cli tools to talk to it. > >>>> I don't understand where cloudmonkey comes into play. CloudMonkey is a > >>>> cli for the CloudStack API. Unless you plan to integrate the > >>>> cloudformation API directly in the cloudstack source code, the > >>>> integration you propose is not clear to me. > >>>> > >>> > >>> Sebastien is correct. I intend to put in the query API server around > the > >>> core of stack mate soon (as soon as I'm done helping on the internal > >>> loadbalancer). This will be written in Ruby. > >>> > >>> > >> > >> Dharmesh I suggest you propose the following: > >> > >> 1-Deploy CloudStack and understand instance > configuration/contextualization > >> 2-Test and deploy Mesos on a set of CloudStack based VM, manually. > >> Design/propose an automation framework. > >> 3-Test stackmate and engage chiradeep (report bugs, make suggestion, > make > >> pull request) > >> 4-Create cloud formation template to provision a Mesos Cluster > >> 5-Compare with Apache Whirr or other cluster provisioning tools. > >> 6-Potentially if you see a link with cloudmonkey, see how you could > extend > >> it to talk to stackmate in a similar manner that it talks to CloudStack. > >> > >> > >> You are pretty close and this is a very exciting projects, so go ahead, > >> modify a bit your proposal and submit it. > >> > >> Deadline for applications is this Friday May 3rd. > >> > >> -sebastien > >> > >> > >> > >> > >
