Aurora is aimed at long-running stateless services (like app servers)? -Brian
On Mon, Aug 26, 2013 at 3:27 PM, Dave Lester <d...@ischool.berkeley.edu>wrote: > Hi All, > > We're pleased to share a draft ASF incubation proposal for Aurora, a > service scheduler used to schedule jobs onto Apache Mesos that we've > developed at Twitter. Aurora provides all of the primitives necessary to > quickly deploy and scale stateless and fault tolerant services in a > datacenter. The complete proposal can be found: > https://wiki.apache.org/incubator/AuroraProposal, and also pasted below. > > In particular, we'd love to add additional mentors to the project. Your > feedback is appreciated. > > Dave > > = Abstract = > > Aurora is a service scheduler used to schedule jobs onto Apache Mesos. > > = Proposal = > > Aurora is a scheduler that provides all of the primitives necessary to > quickly deploy and scale stateless and fault tolerant services in a > datacenter. > > Aurora builds on top of Apache Mesos and provides common features that > allow any site to run large scale production applications. While the > project is currently used in production at Twitter, we wish to develop a > community to increase contributions and see it thrive in the future. > > = Background = > > The initial development of Aurora was done at Twitter, and is planned to be > open sourced. This proposal is for Aurora to join the Apache Incubator. > > = Rationale = > > While the Apache Mesos core focuses on distributing individual tasks across > nodes in a cluster, typical services consist of dozens or hundreds of > replicas of tasks. As a service scheduler, Aurora provides the abstraction > of a "job" to bundle and manage these tasks. Aurora provides many key > functionalities centered around a job, including: definition, the concept > of an instance and the serverset, deployment and scheduling, health > checking, and introspection. It also allows cross-cutting concerns to be > handled like observability and log collection. > > = Current Status = > > == Meritocracy == > > By submitting this incubator proposal, we’re expressing our intent to build > a diverse developer community around Aurora that will conduct itself > according to The Apache Way and use meritocratic means of accepting > contributions. Several members of the Aurora team overlap with Apache > Mesos, which successfully graduated from the Incubator and has embraced a > meritocratic model of governance; we plan to follow a similar path forward > with Aurora and believe that a synergy between both projects will make this > even easier. > > == Community == > > Aurora is currently being used internally at Twitter. By open sourcing the > project, we hope to extend our contributor base significantly and create a > vibrant community around the project. > > == Core Developers == > > Aurora is currently being developed by a team of seven engineers at > Twitter. > > == Alignment == > > The ASF is a natural choice to host the Aurora project, given the goal of > open sourcing the project and fostering a community to grow and support the > software. Additionally, Aurora integrates with Apache Mesos, and Apache > ZooKeeper for service discovery. > > We believe that inclusion within Apache will build stronger ties between > these projects, and create further alignment between their goals and > communities. > > = Known Risks = > > == Orphaned Products == > > The core developers plan to continue working full time on the project, and > there is very little risk of Aurora being abandoned since it is running > hundreds of services as part of Twitter’s infrastructure. Additionally, > members of the Mesos community beyond Twitter have expressed interest in an > advanced scheduler like Aurora (see “Interested Parties” section); we > believe that need will drive some of the community involvement necessary > for the project to incubate successfully. > > == Inexperience with Open Source == > > Initial Aurora committers have varying levels of experience using and > contributing to Open Source projects, however by working with our mentors > and the Apache community we believe we will be able to conduct ourselves in > accordance with Apache Incubator guidelines. The close relationship between > the Aurora team and Apache Mesos means there is an awareness of the > incubation process and a willingness to embrace The Apache Way. > > == Homogenous Developers == > > The initial set of committers are from a single organization, however we > expect that once approved for incubation the project will attract > contributors from more organizations. We have already had conversations > with other companies who have expressed an interest in Aurora. > > == Reliance on Salaried Developers == > > Initial Aurora committers are salaried developers at Twitter, however > shortly after open sourcing the code we plan to diversify the project’s > core committers and contributors. > > == Relationships with Other Apache Products == > > Initially, Aurora has been developed as a scheduler for Apache Mesos. > Additionally, it relies on ZooKeeper for service discovery, allowing > servers to register at a location and clients to subsequently discover the > servers. > > == An Excessive Fascination with the Apache Brand == > > While we respect the reputation of the Apache brand and have no doubts that > it will attract contributors and users, our interest is primarily to give > Aurora a solid home as an open source project following an established > development model. We have also given reasons in the Rationale and > Alignment sections. > > = Documentation = > > This proposal exists online as > http://wiki.apache.org/incubator/AuroraProposal. Project documentation > exists and will be released at the same time as an initial code push. > > = Initial Source = > > JIRA Aurora (AURORA) > > Source and Intellectual Property Submission Plan > The current codebase is available upon request. > > = External Dependencies = > > All Aurora dependencies have Apache compatible licenses, except cron4j > (LGPL). An alternative library, most likely Quartz (Apache Public License > 2.0) will be used prior to an initial code push. > > = Cryptography = > Not applicable. > > = Required Resources = > > == Mailing Lists == > > * aurora-private for private PMC discussions > * aurora-dev > * aurora-commits > * aurora-user > * aurora-issues > > == Subversion Directory == > > We prefer to use Git as our source control system: git:// > git.apache.org/aurora > > == Issue Tracking == > > JIRA Aurora (AURORA) > > = Initial Committers = > > * William Farner (bill at twitter dot com) > * Brian Wickman (wickman at twitter dot com) > * Kevin Sweeney (ksweeney at twitter dot com) > * Sathya Hariesh (sathya at twitter dot com) > * Jonathan Boulle (jon at twitter dot com) > * Maxim Khutornenko (mkhutornenko at twitter dot com) > * Suman Karumuri (skarumuri at twitter dot com) > * Dave Lester (dlester at twitter dot com) > > = Affiliations = > > Twitter > > = Interested Parties = > > * Airbnb > * Foursquare > * Mesosphere > > = Sponsors = > > == Champion == > > Ben Hindman > > == Nominated Mentors == > > * Chris Mattmann > * Ben Hindman > > == Sponsoring Entity == > Incubator PMC >
