+1 (non-binding) On Wed, Jun 22, 2016 at 5:41 PM, Dave Rusek <dru...@twitter.com.invalid> wrote:
> +1 > > On Tue, Jun 21, 2016 at 9:49 PM, Mahak Patidar <mah...@gmail.com> wrote: > > > +1 > > On Jun 21, 2016 7:06 PM, "Josh Elser" <els...@apache.org> wrote: > > > > > +1 (binding) > > > > > > Good luck! > > > > > > Sijie Guo wrote: > > > > > >> Hello All, > > >> > > >> Following the discussion thread, I would like to call a VOTE on > > accepting > > >> DistributedLog into the Apache Incubator. > > >> > > >> [] +1 Accept DistributedLog into the Apache Incubator > > >> [] +0 Abstain. > > >> [] -1 Do not accept DistributedLog into the Apache Incubator because > ... > > >> > > >> This vote will be open for at least 72 hours. > > >> > > >> The proposal follows, you can also access the wiki page: > > >> https://wiki.apache.org/incubator/DistributedLogProposal > > >> > > >> Here is my +1. > > >> > > >> Thanks, > > >> Sijie > > >> > > >> = Abstract = > > >> DistributedLog is a high-performance replicated log service. It offers > > >> durability, replication and strong consistency, which provides a > > >> fundamental building block for building reliable distributed systems, > > e.g > > >> replicated-state-machines, general pub/sub systems, distributed > > databases, > > >> distributed queues and etc. > > >> > > >> See “Building Distributedlog - Twitter’s high performance replicated > log > > >> service” for details: > > >> > > >> > > > https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-log-service > > >> > > >> = Proposal = > > >> We propose to contribute DistributedLog codebase and associated > > artifacts > > >> (e.g. documentation, web-site content etc.) to the Apache Software > > >> Foundation with the intent of forming a productive, meritocratic and > > open > > >> community around DistributedLog’s continued development, according to > > the > > >> ‘Apache Way’. > > >> > > >> = Background = > > >> Engineers at Twitter began developing DistributedLog in early 2013. > > >> DistributedLog is described in a Twitter engineering blog post and > > >> presented at the Messaging Meetup in Sep 2015. It has been released as > > an > > >> Apache-licensed open-source project on GitHub in May 2016. > > >> > > >> DistributedLog is a high-performance replicated log service, which > > >> provides > > >> simple stream-oriented abstractions over log-segments and offers > > >> durability, replication and strong consistency for building reliable > > >> distributed systems. The features offered by DistributedLog includes: > > >> > > >> * Simple high-level, stream oriented interface > > >> * Naming and metadata scheme for managing streams and other entities > > >> * Log data management policies, include data segmentation and data > > >> retention > > >> * Fast write pipeline leveraging batching and compression > > >> * Fast read mechanism leveraging long-poll and read-ahead caching > > >> * Service tiers supporting writer fan-in and reader fan-out > > >> * Geo-replicated logs > > >> > > >> DistributedLog’s most important benefit is high-performance with a > > strong > > >> durability guarantee, making it extremely appropriate for running > > >> different > > >> workloads from distributed database journaling to real-time stream > > >> computing. Its modern, layered architecture makes it easy to run the > > >> service tiers in multi-tenant datacenter environments such as Apache > > Mesos > > >> or cloud environments such as EC2. > > >> > > >> = Rationale = > > >> DistributedLog is designed to provide core fundamental features like > > >> high-performance, durability and strong consistency to anyone who is > > >> building reliable distributed systems, in a simple and efficient way. > > >> > > >> We believe that the ASF is the right venue to foster an open-source > > >> community around DistributedLog’s development. We expect that > > >> DistributedLog will benefit from collaboration with related Apache > > >> projects, and under the auspices of the ASF will attract talented > > >> contributors who will push DistributedLog’s development forward at a > > >> faster > > >> pace. > > >> > > >> We believe that the timing is right for DistributedLog’s development > to > > >> move to the ASF: DistributedLog has already run in production at > Twitter > > >> for 3 years and served various workloads including a distributed > > database > > >> journal, reliable cross datacenter replication, search ingestion, > > >> andgeneral pub/sub messaging. The project is stable. We are excited to > > see > > >> where an ASF-based community can take DistributedLog. > > >> > > >> = Current Status = > > >> DistributedLog is a stable project that has been used in production at > > >> Twitter for 3 years. The source code is public at github.com/twitter, > > >> which > > >> will seed the Apache git repository. > > >> > > >> = Meritocracy = > > >> We understand the central importance of meritocracy to the Apache Way. > > We > > >> will work to establish a welcoming, fair and meritocratic community. > > >> Several companies have already expressed interest in this project, and > > we > > >> intend to invite additional developers to participate. We look forward > > to > > >> growing a rich user and developer community. > > >> > > >> = Community = > > >> There is a large need for a performant replicated log service for > > >> applications such as distributed databases, distributed transactional > > >> systems, replicated-state-machines and pub/sub messaging/queuing. We > > want > > >> to attract more developers to the project, and we believe that the > ASF’s > > >> open and meritocratic philosophy will help us with this. We note the > > >> success of other similar projects already part of the ASF, like Kafka. > > >> > > >> = Core Developers = > > >> DistributedLog is actively developed within Twitter. Most of the > > >> developers > > >> are from Twitter. Many of them are committers or PMC members of Apache > > >> BookKeeper. Others aren’t currently affiliated with ASF so they will > > >> require new ICLAs. > > >> > > >> = Alignment = > > >> DistributedLog is related to several other Apache projects: > > >> > > >> * DistributedLog stores log segments as Ledgers in Apache > BookKeeper. > > >> * DistributedLog uses Apache ZooKeeper for naming and metadata > > >> management > > >> and tracking the ownership of logs. > > >> * DistributedLog uses Apache Thrift as its RPC and serialization > > >> framework. > > >> * In the long-term, DistributedLog’s data will be stored in Apache > > >> Hadoop > > >> clusters powered by HDFS filesystem for archives and backup. > > >> > > >> = Known Risks = > > >> == Orphaned Products == > > >> DistributedLog is used as the fundamental messaging infrastructure at > > >> Twitter. It has been serving production traffic for online database > > >> systems, search ingestion and a general pub/sub system. Twitter > remains > > >> committed to developing and supporting the project. Twitter has a > strong > > >> track record in standing behind projects that were contributed to the > > ASF > > >> by its employees, including Apache Mesos, Apache Aurora, Apache > > >> BookKeeper, > > >> Apache Hadoop. There are many companies are interested in using it in > > >> production. > > >> > > >> == Inexperience with Open Source == > > >> The core developers of DistributedLog are committers of Apache > > BookKeeper. > > >> Although other committers on the initial list are committers or have > > less > > >> experience with the ASF, they already are active in Apache BookKeeper > > >> community. We are confident that the project can be run in accordance > > with > > >> Apache principles on an ongoing basis. > > >> > > >> == Homogeneous Developers == > > >> The initial committers are from Twitter. We hope to encourage > > >> contributions > > >> from other developers and grow them into committers after they have > had > > >> time to continue their contributions. > > >> > > >> == Reliance on Salaried Developers == > > >> Many of DistributedLog’s initial set of committers work full-time on > > >> DistributedLog, and are paid to do so. However, as mentioned > elsewhere, > > we > > >> anticipate growth in the developer community which we hope will > include > > >> people from industry, hobbyists, and academics who have an interested > in > > >> distributed messaging systems. > > >> > > >> == Relationships with Other Apache Products == > > >> DistributedLog uses Apache BookKeeper to store log segments and Apache > > >> ZooKeeper to store log metadata and manage log namespaces. It provides > > an > > >> end-to-end solution for replicated logs, to make building reliable > > >> distributed systems much easier. Unlike Kafka or ActiveMQ, > > DistributedLog > > >> is not a full-fledged pub/sub, queuing or messaging system. Instead, > it > > >> is > > >> targeting on providing a fundamental building block for other > > distributed > > >> systems, offering durability, replication and consistency. So it could > > be > > >> used by other distributed systems, such as transactional log for > > >> replicated > > >> state machines (e.g., HDFS NameNode), WAL for distributed databases > > (e.g. > > >> HBase), Journal for in-memory services (e.g., Kestrel) and even > storage > > >> backend for a full-fledged messaging system. > > >> > > >> == An Excessive Fascination with the Apache Brand == > > >> DistributedLog builds on two existing top-level projects, Apache > > >> BookKeeper > > >> and Apache ZooKeeper. Some of the core developers actively participate > > in > > >> both projects and understand well the implications of being hosted by > > >> Apache. We would like this project to build on the same core values of > > ASF > > >> and to grow a community based on meritocracy. Also, there are several > > >> other > > >> projects already hosted by ASF in this space of reliable messaging and > > >> that > > >> overlap with DistributedLog in interests and scope. Consequently, the > > >> combination of all these observations makes us believe that > > DistributedLog > > >> should be hosted by the ASF. > > >> > > >> = Documentation = > > >> Building DistributedLog: Twitter’s high performance replicated log > > >> service ( > > >> > > >> > > > https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-log-service > > >> ) > > >> > > >> Documentation located in http://distributedlog.io. > > >> > > >> = Initial Source = > > >> DistributedLog’s initial source contribution will come from > > >> http://github.com/twitter/distributedlog/. > > >> > > >> = External Dependencies = > > >> DistributedLog depends upon a number of third-party libraries, which > we > > >> list below. > > >> > > >> * Apache BookKeeper (Apache Software License v2.0) > > >> * Apache Commons (Apache Software License v2.0) > > >> * Apache Maven (Apache Software License v2.0) > > >> * Apache Thrift (Apache Software License v2.0) > > >> * Apache ZooKeeper (Apache Software License v2.0) > > >> * Google Guava (Apache Software License v2.0) > > >> * Mockito (MIT License) > > >> * Junit (Eclipse Public License 1.0) > > >> * LZ4-java (Apache Software License v2.0) > > >> * SLF4J (MIT License) > > >> * Twitter Finagle (Apache Software License v2.0) > > >> * Twitter Scrooge (Apache Software License v2.0) > > >> * Twitter Util (Apache Software License v2.0) > > >> > > >> = Required Resources = > > >> We request that following resources be created for the project to use: > > >> > > >> == Mailing lists == > > >> * priv...@distributedlog.incubator.apache.org (moderated > > subscriptions) > > >> * comm...@distributedlog.incubator.apache.org > > >> * d...@distributedlog.incubator.apache.org > > >> * u...@distributedlog.incubator.apache.org > > >> > > >> == Git repository == > > >> https://git.apache.org/distributedlog.git > > >> > > >> == JIRA instance == > > >> JIRA project DLOG (DLOG or DL) > > >> > > >> = Initial Committers = > > >> * Sijie Guo (Apache BookKeeper Committer, Twitter) > > >> * Robin Dhamankar (Apache BookKeeper Committer) > > >> * Leigh Stewart (Twitter) > > >> * Dave Rusek (Twitter) > > >> * Honggang Zhang (Twitter) > > >> * Jordan Bull (Twitter) > > >> * Satish Kotha (Twitter) > > >> * Aniruddha Laud > > >> * Franck Cuny (Twitter) > > >> * Eitan Adler (Twitter) > > >> > > >> == Affiliations == > > >> Most of the initial committers are employees of Twitter, except Robin > > >> Dhamankar and Aniruddha Laud. > > >> > > >> = Sponsors = > > >> == Champion == > > >> Flavio Junqueira > > >> > > >> == Nominated Mentors == > > >> * Flavio Junqueira > > >> * Chris Nauroth > > >> * Henry Saputra > > >> > > >> = Sponsoring Entity = > > >> We ask that the Apache Incubator PMC to sponsor this proposal. > > >> > > >> > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > >
