Hi Owen, Thanks for details explanation about the statement. Really appreciate the clarification.
- Henry On Wed, Jul 16, 2014 at 2:46 PM, Owen O'Malley <omal...@apache.org> wrote: > On Wed, Jul 16, 2014 at 11:29 AM, Henry Saputra <henry.sapu...@gmail.com> > wrote: > >> HI, >> >> Just curious if you could give reference to "Apache encourages >> disjoint teams to form independent projects, even when those projects >> overlap in scope" statement ? >> > > Henry, > It generally comes from the "Community over Code," which is a phrase > discussing how the focus is on how the people working on a project work > together and not the code. The assumption is teams with healthy communities > will build good code. > > In terms of documentation, it like most of Apache is learned by doing. In > Nick Burch's talk "The Apache Way" at ApacheCon this year ( > http://events.linuxfoundation.org/sites/events/files/slides/TheApacheWay14.pdf) > there was a slide about this: > > Don't pick winners, pick runners > • Board doesn't say “We want X” > • Developers say “X is cool” > • We enable developers to do cool stuff > • Apache developers are at the forefront of innovation > • Not interested in a single runner > • We want relay teams > • Community is critical to the Apache Way > • Apache is about supporting communities > > .. Owen > > >> - Henry >> >> On Mon, Jul 14, 2014 at 7:16 PM, Selvamohan Neethiraj >> <sneethi...@hortonworks.com> wrote: >> > Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal) >> > >> > == Abstract == >> > >> > Argus is a framework to enable, monitor and manage comprehensive data >> security across the Hadoop platform. >> > >> > The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in >> Greek mythology, endowed with a role to keep “an eye” open and be an >> effective watchman at all times. >> > >> > == Background == >> > >> > The vision with Argus is to provide comprehensive security across the >> Apache Hadoop ecosystem. With the advent of Apache YARN, the Hadoop >> platform can now support a true data lake architecture. Enterprises can >> potentially run multiple workloads, in a multi tenant environment. Data >> security within Hadoop needs to evolve to support multiple use cases for >> data access, while also providing a framework for central administration of >> security policies and monitoring of user access. >> > >> > XA Secure, a Hadoop security focused startup, developed the initial >> technology behind Argus. XA Secure was acquired by Hortonworks, which now >> is contributing the technology to the open source community to extend and >> innovate. >> > >> > == Rationale == >> > >> > Many of the projects in the Hadoop ecosystem have their own >> authentication, authorization, and auditing components. There are no >> central administration and auditing capabilities. We are looking to address >> these enterprises security needs of central administration and >> comprehensive security through the Argus project. >> > Our initial focus would be around authorization and auditing, the longer >> term vision would be to tie all aspects around data security within the >> Hadoop platform. >> > >> > == Proposal Details == >> > >> > The vision of Argus is to enable comprehensive data security across the >> Hadoop platform. The goal is provide a single user interface or API to >> manage security policies, monitor user access and policy changes history. >> The framework would work with individual components in enforcing these >> policies and in capturing relevant audit information. >> > Initial Goals >> > 1. Donate the Argus source code and documentation to the >> Apache Software Foundation >> > 2. Setup and standardize the open governance of the Argus >> project >> > 3. Build a user and developer community >> > 4. Deeper Integration with Hadoop Platform >> > a. Enable integration with Apache Storm, Apache >> Knox and Apache Falcon for authorization and auditing >> > 5. Configurable centralized storage of audit data into HDFS >> > 6. Enable framework to be run in both Linux and Windows >> environments >> > 7. Rationalize install procedure, making it easier for >> enterprises to deploy >> > >> > == Longer Term Goals == >> > >> > In longer term, Argus should provide a comprehensive security framework >> for Hadoop platform components, covering the following >> > 1. Centralized security administration to manage all >> security related tasks in a central UI >> > 2. Fine grained authorization to do a specific action >> and/or operation with Hadoop component/tool and managed through a central >> administration tool >> > a. Standardize authorization method across all >> Hadoop components >> > b. Enhanced support for different authorization >> methods - Role based access control, attribute based access control etc >> > c. Enable tag based global policies >> > 3. Centralize auditing of user access and administrative >> actions (security related) within all the components of Hadoop >> > >> > == Current Status == >> > >> > Argus’ technology is currently being used by enterprises and is under >> active development. >> > >> > The key components of Argus are: >> > • Enterprise Security Administration Portal >> > ◦ A Java Web Application, designed for >> administration of security policies from a single location for the entire >> hadoop cluster (and even multiple hadoop clusters) >> > • Security Agents >> > ◦ A light-weight Java Agent, which will be >> embedded into the hadoop component (e.g. Hive, HBase and Hadoop) as an >> authorization provider to enforce the security policies and also collect >> access events/logs. >> > • User/Group Synchronizer Module >> > ◦ A standalone daemon which allows the user/group >> information to be synched from the enterprise user repositories like >> LDAP/AD to Argus local database. This user/group information in Argus local >> database will help the security policy administrators >> > ▪ to define security policies by >> selecting users/groups from a drop-down box (instead of typing their >> name/group in a text-box). >> > ▪ to delegate policy administration to >> other users/groups >> > ▪ to restrict view of reports based on the >> users/groups >> > • Centralized Audit Logs and Monitoring >> > ◦ Log events to central data storage/database >> > ◦ Interactive query of audit events >> > ◦ Audit administrator actions >> > >> > >> > The initial version provides ability to >> > 1. Define security policies using a central security >> administration UI. >> > 2. Fine grained access control for HDFS (file level) , Hive >> (column level) and HBase (column level) >> > 3. Framework to record access/operational events/logs as >> part of auditing and view using a central monitoring UI. >> > 4. Support delegated policy administration >> > 5. Centralized audit monitoring and query capabilities >> > >> > === Meritocracy === >> > >> > We plan to invest in supporting a meritocracy. We will discuss the >> requirements in an open forum. Several companies have already expressed >> interest in this project, and we intend to invite additional developers to >> participate. We will encourage and monitor community participation so that >> privileges can be extended to those that contribute. >> > >> > === Community === >> > >> > We are happy to report that there are existing Apache committers and >> corporate users who are closely involved in the project already. We hope to >> extend the user and developer base further in the future and build a solid >> open source community around Argus, growing the community and adding >> committers following the Apache meritocracy model. >> > >> > === Core Developers === >> > >> > The initial technology within Argus was originally built by the team at >> XA Secure. XA Secure was founded and managed by experienced members with a >> wide background in enterprise security. Some of the XA’s core team have >> been proposed as core developers for this project. The developer list also >> include an Apache member and PMC members from several Apache projects >> (Hadoop, HBase, and Knox). A concern is that all of the core developers are >> employed by Hortonworks and thus an emphasis will be on increasing the >> diversity of the developer community. >> > Alignment >> > The initial committers strongly believe that a unified security portal >> for Apache Hadoop, Hive, and HBase will gain broad adoption as an open >> source, community driven project. Our hope is that the Apache Falcon, >> Apache Storm, Apache Knox, and other communities will find tremendous >> value in Argus and will adopt it en masse. >> > >> > == Known Risks == >> > >> > === Orphaned Products === >> > >> > The initial code behind Argus is under active development and is being >> actively used by several enterprises. It is not expected to be orphaned. >> > >> > === Inexperience with Open Source === >> > >> > Many of the core developers have long-standing experience in open >> source, Dili Aramugam, Kevin Minder and Larry McCay are committers on the >> Apache Knox project. Sanjay Radia and Owen O’Malley are PMC members on >> several Apache projects. We have several mentors that will work with the >> inexperienced committers on building a thriving developer community. >> > >> > === Homogeneous Developers === >> > The current core developers are all from Hortonworks. However, we expect >> to establish a thriving developer community that includes users of Argus >> and developers of other Hadoop components. >> > >> > === Reliance on Salaried Developers === >> > >> > Currently, all of the developers are paid to work on Argus. A key goal >> for the incubation process will be to broaden the developer base. >> > Relationships with Other Apache Products >> > The biggest risk is fast rate of growth of new features within the >> Hadoop ecosystem and security standards not being applied during the >> initial development of these new products. We believe an active engagement >> from the Hadoop community would significantly aid adoption of common >> security framework across the ecosystem and will help in establishing cross >> component standards. >> > >> > As mentioned in the Alignment section, Argus is closely integrated with >> Hadoop, Hive and HBase in a numerous ways. We look forward to collaborating >> with those communities, as well as other Apache communities. >> > >> > There is some overlap between the goals of Argus and Apache Sentry. >> Apache encourages disjoint teams to form independent projects, even when >> those projects overlap in scope. Additionally, we feel that the distinct >> code bases, development teams, and different approaches to the problem >> should be represented by different projects. This will provide better >> choices for users to choose from. >> > >> > === An Excessive Fascination with the Apache Brand === >> > >> > While we respect the reputation of the Apache brand and have no doubts >> that it will attract contributors and users, our interest is primarily to >> give Argus a solid home as an open source project with a broad developer >> base and to encourage adoption by the related ASF projects and foster >> innovation around security >> > >> > == Documentation == >> > >> http://hortonworks.com/blog/hortonworks-acquires-xasecure-to-provide-comprehensive-security-for-enterprise-hadoop/ >> > >> > == Initial Source == >> > >> > We will make the initial source available as a patch. >> > >> > == Source and IP Submission Plan == >> > 1. All source will be moved to Apache Infrastructure >> > 2. All outstanding issues in our in-house JIRA infrastructure will >> be replicated into the Apache JIRA system. >> > 3. We will be acquiring a twitter handle for project Argus (eg: >> @apacheargus ) >> > >> > == External Dependencies == >> > >> > Argus has no external dependencies except for some Java libraries that >> are considered ASF-compatible (JUnit, SLF4J, …) and Apache artifacts : >> Hadoop, Log4J and the transient dependencies of all these artifacts. >> > >> > == Cryptography == >> > >> > Argus does not incorporate encryption currently. >> > >> > == Required Resources == >> > >> > === Mailing Lists: === >> > 1. argus-dev >> > 2. argus-commits >> > 3. argus-private >> > >> > === Infrastructure: === >> > 1. Git repository >> > 2. JIRA Argus >> > 3. Gerrit for reviewing patches >> > The existing code includes local host integration tests, so we would >> like a Jenkins instance to run them whenever a new patch is submitted. >> > >> > == Initial Committers == >> > >> > * Balaji Ganesan (bganesan at hortonworks.com) >> > * Dilli Arumugam (darumugam at hortonworks.com) >> > * Don Bosco Durai (bdurai at hortonworks.com) >> > * Kevin Minder (kminder at apache.org) >> > * Larry McCay (lmccay at apache.org) >> > * Madhanmohan Neethiraj (mneethiraj at hortonworks.com) >> > * Owen O’Malley (omalley at apache.org) >> > * Ramesh Mani (rmani at hortonworks.com) >> > * Sanjay Radia (sradia at apache.org) >> > * Selvamohan Neethiraj (sneethiraj at hortonworks.com) >> > >> > == Affiliations == >> > >> > * Balaji Ganesan - Hortonworks >> > * Dilli Arumugam - Hortonworks >> > * Don Bosco Durai - Hortonworks >> > * Kevin Minder - Hortonworks >> > * Larry McCay - Hortonworks >> > * Madhanmohan Neethiraj - Hortonworks >> > * Owen O’Malley - Hortonworks >> > * Ramesh Mani - Hortonworks >> > * Sanjay Radia - Hortonworks >> > * Selvamohan Neethiraj - Hortonworks >> > >> > == Sponsors == >> > >> > === Champion: === >> > >> > * Owen O’Malley (omalley at apache.org) - Hortonworks >> > >> > === Nominated Mentors: === >> > >> > * Alan Gates - Hortonworks >> > * Devaraj Das - Hortonworks >> > * Jakob Homan - LinkedIn >> > * Owen O’Malley - Hortonworks >> > >> > === Sponsoring Entity === >> > >> > Incubator PMC >> > >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> > For additional commands, e-mail: general-h...@incubator.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
