Random lurker project name suggestion: yarnbomb - see < http://en.m.wikipedia.org/wiki/Yarn_bombing>
/me relurking Steve On Oct 30, 2013 2:00 PM, "Donald Whytock" <dwhyt...@apache.org> wrote: > Knit, crochet, macrame? > > Though, honestly, pulling lower-level components together into higher-level > ones sounds a little like granny squares... > > Don > > > On Wed, Oct 30, 2013 at 1:53 AM, Andreas Neumann <a...@apache.org> wrote: > > > Pardon my ignorance, I just looked it up and realized that warp and weft > > are indeed related to weaving, so they might work. > > I do have the impression, though, that most people would associate Warp > > with the speed of light and not with weaving. > > > > -Andreas. > > > > On Wed, Oct 30, 2013 at 1:41 AM, Andreas Neumann <a...@apache.org> > wrote: > > > > > Thanks for pointing out these similarities; we were not aware of > Commons > > > Weaver. Given that Weaver is a sub-project of Commons, would the > > similarity > > > be tolerable? Also, since Weave and Wave are pronounced quite > > differently, > > > I am hoping that they are perceived as different enough. > > > > > > The name Weave is motivated from the name of YARN - it takes the > > > complexity out of yarn by "weaving" it into a simple pattern. Warp and > > Weft > > > don't really convey this meaning. > > > > > > If the concern about the name similarity is really strong, we will try > to > > > find another name, and we are open to suggestions. But please do also > > > consider the motivation for this naming. > > > > > > Thanks > > > > > > > > > > > > On Tue, Oct 29, 2013 at 7:54 PM, sebb <seb...@gmail.com> wrote: > > > > > >> In which case, maybe consider the related words: > > >> > > >> Apache Warp > > >> Apache Weft > > >> > > >> Just a thought. > > >> > > >> On 29 October 2013 22:14, Upayavira <u...@odoko.co.uk> wrote: > > >> > And Apache Wave too (which is what I first saw before I read the > title > > >> > more carefully). > > >> > > > >> > Upayavira > > >> > > > >> > On Tue, Oct 29, 2013, at 09:12 PM, Matt Benson wrote: > > >> >> Hi, > > >> >> I am concerned about potential confusion with Apache Commons > Weaver > > >> >> [1]. > > >> >> > > >> >> Matt > > >> >> > > >> >> [1] https://commons.apache.org/proper/commons-weaver/ > > >> >> > > >> >> > > >> >> On Tue, Oct 29, 2013 at 2:53 PM, Andreas Neumann <a...@apache.org> > > >> wrote: > > >> >> > > >> >> > I would like to propose Weave, an abstraction over Apache Hadoop® > > >> YARN to > > >> >> > reduce the complexity of developing distributed applications, as > an > > >> >> > Apache Incubator > > >> >> > podling. > > >> >> > > > >> >> > The proposal is included in plain text. I would also like to put > > >> this on > > >> >> > the wiki, but I appear to lack privileges to create pages. What > do > > I > > >> need > > >> >> > to do to get permission? > > >> >> > > > >> >> > -Andreas. > > >> >> > > > >> >> > Abstract > > >> >> > ======== > > >> >> > > > >> >> > Weave is an abstraction over Apache Hadoop® YARN that reduces the > > >> >> > complexity of developing distributed applications, allowing > > >> developers to > > >> >> > focus more on their business logic. > > >> >> > > > >> >> > Proposal > > >> >> > ======== > > >> >> > > > >> >> > Weave is a set of libraries that reduces the complexity of > > developing > > >> >> > distributed applications. It exposes the distributed capabilities > > of > > >> Apache > > >> >> > Hadoop® YARN via a simple and intuitive programming model similar > > to > > >> Java > > >> >> > threads. Weave also has built-in capabilities required by many > > >> distributed > > >> >> > applications, such as real-time application logs and metrics > > >> collection, > > >> >> > application lifecycle management, and network service discovery. > > >> >> > > > >> >> > Background > > >> >> > ========== > > >> >> > > > >> >> > Hadoop YARN is a generic cluster resource manager that supports > any > > >> type of > > >> >> > distributed application. However, YARN’s interfaces are too low > > >> level for > > >> >> > rapid application development. It requires a great deal of > > >> boilerplate code > > >> >> > even for a simple application, creating a high ramp up cost that > > can > > >> turn > > >> >> > developers away. > > >> >> > > > >> >> > Weave is designed to improve this situation with a programming > > model > > >> that > > >> >> > makes running distributed applications as easy as running Java > > >> threads. > > >> >> > With the abstraction provided by Weave, applications can be > > executed > > >> in > > >> >> > process threads during development and unit testing and then be > > >> deployed to > > >> >> > a YARN cluster without any modifications. > > >> >> > > > >> >> > Weave also has built-in support for real-time application logs > and > > >> metrics > > >> >> > collection, delegation token renewal, application lifecycle > > >> management, and > > >> >> > network service discovery. This greatly reduces the pain that > > >> developers > > >> >> > face when developing, debugging, deploying and monitoring > > distributed > > >> >> > applications. > > >> >> > > > >> >> > Weave is not a replacement for YARN, it’s a framework that > operates > > >> on top > > >> >> > of YARN. > > >> >> > > > >> >> > Rationale > > >> >> > ========= > > >> >> > > > >> >> > Developers who write YARN applications typically find themselves > > >> >> > implementing the same (or similar) boilerplate code over and over > > >> again for > > >> >> > every application. It makes sense to distill this common code > into > > a > > >> >> > reusable set of libraries that is perpetually maintained and > > >> improved by a > > >> >> > diverse community of developers. > > >> >> > > > >> >> > Weave’s simple thread-like programming model will enable many > Java > > >> >> > programmers to develop distributed applications. We believe that > > this > > >> >> > simplicity will attract developers who would otherwise be > > >> discouraged by > > >> >> > complexity, and many new use cases will emerge for the usage of > > YARN. > > >> >> > > > >> >> > Incubating Weave as an Apache project makes sense because Weave > is > > a > > >> >> > framework built on top of YARN, and Weave uses Apache Zookeeper, > > >> HDFS, > > >> >> > Kafka, and other Apache software (see the External Dependencies > > >> section). > > >> >> > > > >> >> > Current Status > > >> >> > ============== > > >> >> > > > >> >> > Weave was initially developed at Continuuity. The Weave codebase > is > > >> >> > currently hosted in a public repository at github.com, which > will > > >> seed the > > >> >> > Apache git repository. > > >> >> > > > >> >> > Meritocracy > > >> >> > ----------- > > >> >> > Our intent with this incubator proposal is to start building a > > >> diverse > > >> >> > developer community around Weave following the Apache meritocracy > > >> model. > > >> >> > Since Weave was initially developed in early 2013, we have had > fast > > >> >> > adoption and contributions within Continuuity. We are looking > > >> forward to > > >> >> > new contributors. We wish to build a community based on Apache's > > >> >> > meritocracy principles, working with those who contribute > > >> significantly to > > >> >> > the project and welcoming them to be committers both during the > > >> incubation > > >> >> > process and beyond. > > >> >> > > > >> >> > Community > > >> >> > --------- > > >> >> > Weave is currently being used internally at Continuuity and is at > > >> the core > > >> >> > of our products. We hope to extend our contributor base > > >> significantly and > > >> >> > we will invite all who are interested in simplifying the > > development > > >> of > > >> >> > distributed applications to participate. > > >> >> > > > >> >> > Core Developers > > >> >> > --------------- > > >> >> > Weave is currently being developed by five engineers at > > Continuuity: > > >> >> > Terence Yim, Andreas Neumann, Gary Helmling, Poorna Chandra and > > >> Albert > > >> >> > Shau. > > >> >> > Terence Yim is an Apache committer for Helix, Andreas is an > Apache > > >> >> > committer and PMC member for Oozie, and Gary Helmling is an > Apache > > >> >> > committer and PMC member for HBase. Poorna Chandra and Albert > Shau > > >> have > > >> >> > made many contributions to Weave. > > >> >> > > > >> >> > Alignment > > >> >> > --------- > > >> >> > The ASF is the natural choice to host the Weave project as its > goal > > >> of > > >> >> > encouraging community-driven open source projects fits with our > > >> vision for > > >> >> > Weave. > > >> >> > > > >> >> > Additionally, many other projects with which we are familiar and > > >> expect > > >> >> > Weave to integrate with, such as ZooKeeper, YARN, HDFS, log4j, > and > > >> others > > >> >> > mentioned in the External Dependencies section are Apache > projects, > > >> and > > >> >> > Weave will benefit by close proximity to them. > > >> >> > > > >> >> > Known Risks > > >> >> > =========== > > >> >> > > > >> >> > Orphaned Products > > >> >> > ----------------- > > >> >> > There is very little risk of Weave being orphaned, as it is a key > > >> part of > > >> >> > Continuuity’s products. The core Weave developers plan to > continue > > >> to work > > >> >> > on Weave, and Continuuity has funding in place to support their > > >> efforts > > >> >> > going forward. > > >> >> > Many other Big Data companies can benefit from Weave, and we have > > >> already > > >> >> > received interest from various entities that would like to use > and > > >> >> > contribute to Weave. > > >> >> > > > >> >> > Inexperience with Open Source > > >> >> > ----------------------------- > > >> >> > Several of the core developers have experience with open source > > >> >> > development. Terence Yim, Andreas Neumann and Gary Helmling are > > >> currently > > >> >> > Apache committers for Helix, Oozie and HBase respectively > > >> >> > Homogeneous Developers > > >> >> > > > >> >> > The current core developers are all Continuuity employees. > However, > > >> we > > >> >> > intend to establish a developer community that includes > independent > > >> and > > >> >> > corporate contributors. We are encouraging new contributors via > our > > >> mailing > > >> >> > lists, public presentations, and personal contacts, and we will > > >> continue to > > >> >> > do so. Various entities have already expressed interest in > becoming > > >> >> > involved with Weave. > > >> >> > > > >> >> > Reliance on Salaried Developers > > >> >> > ------------------------------- > > >> >> > Currently, these developers are paid to work on Weave. Once the > > >> project has > > >> >> > built a community, we expect to attract committers, developers > and > > >> >> > community other the current core developers. However, because > > >> Continuuity > > >> >> > products use Weave internally, the reliance on salaried > developers > > is > > >> >> > unlikely to change, at least in the near term. > > >> >> > > > >> >> > Relationships with Other Apache Products > > >> >> > ---------------------------------------- > > >> >> > Weave is deeply integrated with Apache projects. Weave uses > Apache > > >> YARN as > > >> >> > its underlying resource management and task scheduling system and > > >> Apache > > >> >> > Zookeeper for coordination. In addition, Weave uses Apache HDFS > and > > >> Apache > > >> >> > Kafka. A number of other Apache projects are Weave dependencies > and > > >> are > > >> >> > listed in the External Dependencies section. > > >> >> > > > >> >> > An Excessive Fascination with the Apache Brand > > >> >> > ---------------------------------------------- > > >> >> > While we respect the reputation of the Apache brand and have no > > >> doubt that > > >> >> > it will attract contributors and users, our interest is primarily > > to > > >> give > > >> >> > Weave a solid home as an open source project following an > > established > > >> >> > development model. We have also given additional reasons in the > > >> Rationale > > >> >> > and Alignment sections. > > >> >> > > > >> >> > Documentation > > >> >> > ============= > > >> >> > > > >> >> > https://github.com/continuuity/weave > > >> >> > > > >> >> > Initial Source > > >> >> > ============== > > >> >> > > > >> >> > The Weave codebase is currently hosted at > > >> >> > https://github.com/continuuity/weave. > > >> >> > > > >> >> > External Dependencies > > >> >> > ===================== > > >> >> > > > >> >> > The dependencies all have Apache-compatible licenses: > > >> >> > • avro (Apache 2.0) > > >> >> > • hadoop (Apache 2.0) > > >> >> > • gson (Apache 2.0) > > >> >> > • guava-libraries (Apache 2.0) > > >> >> > • hbase (Apache 2.0) > > >> >> > • hdfs (Apache 2.0) > > >> >> > • kafka (Apache 2.0) > > >> >> > • netty (Apache 2.0) > > >> >> > • snappy-java (Apache 2.0) > > >> >> > • yarn (Apache 2.0) > > >> >> > • zookeeper (Apache 2.0) > > >> >> > • asm (BSD) > > >> >> > • junit (EPL v1.0) > > >> >> > • logback (EPL v1.0 ) > > >> >> > • slf4j (MIT) > > >> >> > > > >> >> > Cryptography > > >> >> > ============ > > >> >> > > > >> >> > Weave will depend on secure Hadoop, which can optionally use > > >> Kerberos. > > >> >> > > > >> >> > Required Resources > > >> >> > ================== > > >> >> > > > >> >> > Mailing Lists > > >> >> > ------------- > > >> >> > - weave-private for private PMC discussions (with moderated > > >> subscriptions) > > >> >> > - weave-dev for technical discussions among contributors > > >> >> > - weave-commits for notification about commits > > >> >> > > > >> >> > Subversion Directory > > >> >> > -------------------- > > >> >> > Git is the preferred source control system: git:// > > >> git.apache.org/weave > > >> >> > > > >> >> > Issue Tracking > > >> >> > -------------- > > >> >> > JIRA Weave (WEAVE) > > >> >> > > > >> >> > Other Resources > > >> >> > --------------- > > >> >> > The existing code already has unit tests, so we would like a > Hudson > > >> >> > instance to run them whenever a new patch is submitted. This can > be > > >> added > > >> >> > after project creation. > > >> >> > > > >> >> > Initial Committers > > >> >> > ================== > > >> >> > > > >> >> > • Terence Yim > > >> >> > • Andreas Neumann > > >> >> > • Gary Helmling > > >> >> > • Poorna Chandra > > >> >> > • Albert Shau > > >> >> > > > >> >> > Affiliations > > >> >> > ============ > > >> >> > > > >> >> > • Terence Yim (Continuuity) > > >> >> > • Andreas Neumann (Continuuity) > > >> >> > • Gary Helmling (Continuuity) > > >> >> > • Poorna Chandra (Continuuity) > > >> >> > • Albert Shau (Continuuity) > > >> >> > > > >> >> > Sponsors > > >> >> > ======== > > >> >> > > > >> >> > Champion > > >> >> > -------- > > >> >> > Vinod K <vinodkv at apache dot org> (Apache Member) > > >> >> > > > >> >> > Nominated Mentors > > >> >> > ----------------- > > >> >> > • Arun C Murthy <acmurthy at apache dot org> > > >> >> > • Tom White <tomwhite at apache dot org> > > >> >> > • Patrick Hunt <phunt at apache dot org> > > >> >> > > > >> >> > Sponsoring Entity > > >> >> > ----------------- > > >> >> > We are requesting that the Incubator sponsor this project. > > >> >> > > > >> > > > >> > > --------------------------------------------------------------------- > > >> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > >> > For additional commands, e-mail: general-h...@incubator.apache.org > > >> > > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > >> For additional commands, e-mail: general-h...@incubator.apache.org > > >> > > >> > > > > > >
