Thanks Hitesh, this is great data too. I also found the issues list to be a gold source of conversation.
I'll now wait to see what Ted thinks, but as far as I'm concerned you guys are moving in the direction I suspected (graduation) based on my admittedly limited searching of mail archives and basic metrics done the other day. I'll wait on this for a few days and let the conversation develop. This is the great thing about a DISCUSS thread - discussion happens! :) Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Hitesh Shah <hit...@apache.org> Reply-To: "general@incubator.apache.org" <general@incubator.apache.org> Date: Monday, June 23, 2014 1:20 PM To: "general@incubator.apache.org" <general@incubator.apache.org> Cc: "d...@tez.incubator.apache.org" <d...@tez.incubator.apache.org> Subject: Re: Tez graduation [Was: Request for mentor assessment] >Hi folks, > >To be very clear, the project's source of truth has always been its >status file: >https://svn.apache.org/repos/asf/incubator/public/trunk/content/projects/t >ez.xml > >For the tez website, http://tez.incubator.apache.org/team-list.html, the >onus was for the committers to test out their commit privileges, update >the teamlist in the source tree and publish the updated website. I >understand it has created a lot of confusion for folks outside of the >project. I will go ahead and update the internal list to match the source >of truth. > >Even though I am well aware that some of the initial committers on the >incubator proposal have not been active ( my take on active is as simple >as sending a single mail to any mailing list ) on the project since >inception, my understanding had been that all committers and mentors from >an incubator podling become PMC for the top level project when it >graduates. Is my understanding incorrect? In any case, to directly >address the diversity question, I believe the PMC would have members from >companies such as Hortonworks, Yahoo, LinkedIn, Twitter to name a few. > >As for our community, we have had multiple talks and meetups both on Tez >alone as well as ones with the folks from Hive and Pig. > - http://www.meetup.com/Apache-Tez-User-Group. The only meetup we have >had till date had a detailed talk on Tez in addition to presentations >from folks from Hive and Pig on how they were using or planning to use >Tez. > - There was a whole evening devoted to Tez and applications on Tez as >part of the Bay Area Hadoop User Group - >http://www.meetup.com/hadoop/events/116895522/ > - Talks - there have been quite a few talks on it over the past year. >For example, one can look at the recently completed Hadoop Summit in San >Jose and search for Tez related talks of which only one talk was just >about Tez and the others about applications built on top of it. > >We have been working with the Hive and Pig folks mostly via JIRA to >address their requirements as they built out their platforms to work on >Tez. One can look at https://issues.apache.org/jira/browse/HIVE-4660 and >https://issues.apache.org/jira/browse/PIG-3446 to see the considerable >amount of work both these communities have put in to get their respective >applications to work on top of Tez. Most of the interaction has been on >JIRA and not really on the dev mailing lists. Most design discussions >also tend to happen there. Subscribing to issues@ might give one a better >perspective on community interaction. > >To be frank, the Tez community for the past year has been working towards >helping make Apache Hive and Apache Pig faster. As the platform evolves >and grows, other applications such as Flink are considering it too. >Hopefully, at some point down the line, we would like to work with the >MapReduce community to have them consider using Tez. > >thanks >‹ Hitesh > >On Jun 23, 2014, at 12:50 AM, Ted Dunning <ted.dunn...@gmail.com> wrote: > >> Let me repeat that my desired outcome here is Tez graduation. >> >> It is just that I want to see Tez graduate as a viable project. >> >> >> >> >> On Mon, Jun 23, 2014 at 12:43 AM, Mattmann, Chris A (3980) < >> chris.a.mattm...@jpl.nasa.gov> wrote: >> >>> Thanks Ted, these are all important points that require responses. I >>>have >>> CC'ed the dev@tez.i.a.o >>> list, where I hope the Tez community can respond to your points below >>>in >>> detail. >>> >>> Guys, please see Ted's comments below and please try to address them in >>> light of the >>> >>> recent [DISCUSS] Graduation thread I raised. It's fine if the answer to >>> some of the >>> points is (1) we plan on addressing them by X date, with Y action; (2) >>>we >>> don't think >>> this is a valid point *because* ..*explanation*; (3) you didn't >>>consider >>> this factoid >>> _here_, etc. >>> >>> I'm hoping the other mentors on the project can step up and have some >>> insight >>> into the below. I look forward to the discussion. >>> >>> >>> Cheers, >>> Chris >>> >>> >>> -----Original Message----- >>> From: Ted Dunning <ted.dunn...@gmail.com> >>> Reply-To: "general@incubator.apache.org" <general@incubator.apache.org> >>> Date: Monday, June 23, 2014 12:12 AM >>> To: "general@incubator.apache.org" <general@incubator.apache.org> >>> Subject: Re: Tez graduation [Was: Request for mentor assessment] >>> >>>> On Sun, Jun 22, 2014 at 9:33 AM, Mattmann, Chris A (3980) < >>>> chris.a.mattm...@jpl.nasa.gov> wrote: >>>> >>>>>> Regardless of whether it is an explicit requirement to have some >>>>> specific >>>>>> level of diversity, there is a requirement that the group >>>>>>demonstrate >>>>>> community building. >>>>> >>>>> I'm also sympathetic to this: you can see my thread here where >>>>> originally >>>>> I didn't >>>>> think much had been going on until I explicitly checked the lists and >>>>> went >>>>> through >>>>> the archives and saw ~7 IIRC PPMC members/committers added since >>>>> entering >>>>> [1]. That's >>>>> community building regardless of what affiliation those new members >>>>>came >>>>> from. >>>>> >>>>> The other metrics and statistics (health of the mailing lists/open >>>>> discussions have >>>>> been increasing; releases have been made, etc.) are all there and >>>>> outlined >>>>> in [1]. >>>> >>>> >>>> As I read the private list, there has been absolutely no effort at >>>>adding >>>> committers until last month when the topic of graduation has come up. >>>> There >>>> is no substantive discussion on the list of the potential committers, >>>>but >>>> simply rather perfunctory +1 votes. >>>> >>>> In reading the thread about whether there should be/is a diversity >>>> requirement, I come to a bit of a different conclusion as well. What >>>>I >>>> see >>>> in the discussion is that the participants seem to agree that >>>> >>>> - there should not be a simple count-based diversity measure because >>>>this >>>> would dissuade valuable commercial contributions >>>> >>>> - the chance/likelihood of project failure due to a dominant >>>>commercial >>>> contributor pulling out should not be considered since project >>>>dormancy or >>>> retirement isn't a bad thing. Better to try and fail than never try. >>>> >>>> - there is clear recognition of the risk of commercial companies >>>>inventing >>>> a project not so much because of a desire to build community but >>>>rather as >>>> a marketing maneuver. >>>> >>>> - there is no mention of the problems that have arisen in the Hadoop >>>> groups >>>> relative to use of groups and Apache trademarks for marketing >>>>purposes. >>>> >>>> I don't want to impute motives to actions since I cannot know what >>>>others >>>> hold in their hearts, but I don't see evidence of efforts to really >>>>build >>>> the community. There was one meat-space meetup a year ago. There >>>>have >>>> been a few presentations of what Tez is. >>>> >>>> I do see a fair number of public statements of the form "Tez is >>>> Hortonworks >>>> answer to Impala" or "The future of Hadoop runs on Tez", most of which >>>> seem >>>> to come out of Hortonworks' marketing department. >>>> >>>> In looking at the dev list back to September of last year, I see a >>>>large >>>> number of procedural messages (votes, results, release mechanics), a >>>>few >>>> user questions (NPE in such and so, how do I install) and very few >>>>design >>>> discussions. There was one thread last November about combiners, >>>>another >>>> in January about map parallelism, and one more in March about reducer >>>> parallelism. To me this looks a lot like a project where all design >>>> discussions are occurring off-list. >>>> >>>> What I would like to see would include >>>> >>>> - a strong effort on the part of Tez to bring all design and >>>> implementation >>>> discussions to the list, >>>> >>>> - continuous community engagement efforts such as weekly hangouts in >>>> different timezones to help new contributors understand what is >>>>happening >>>> technically in the project in an interactive setting >>>> >>>> - lots of public talks focussed on how outsiders can contribute and >>>>how >>>> the >>>> design works >>>> >>>> - cross posting on related projects offering cross pollination >>>> opportunities. Such related groups might be Drill, Optiq, Tajo and >>>>Spark. >>>> These postings would say things like "how could you guys help us with >>>> Tez" >>>> or "Tez has this and such, would that be helpful to you guys". >>>> >>>> - inviting contributors from related projects to give technical >>>> presentations in the context of Tez >>>> >>>> Now, my searches have been fairly cursory and could have missed >>>>important >>>> activities, but I have a bit of a hard time believing that I have >>>>missed >>>> major efforts along these lines. >>>> >>>> Based on this evidence, it really does look to me like Tez is a purely >>>> commercial development masquerading as an Apache community. >>>> >>>> I agree that superficial diversity metrics are counter-productive, >>>>but I >>>> also assert that there isn't any obvious evidence of serious community >>>> building here and there is significant evidence that building >>>>community >>>> isn't even the point of the project. That *is* a substantive issue >>>> relative to graduation. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>> For additional commands, e-mail: general-h...@incubator.apache.org >>> >>> > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >For additional commands, e-mail: general-h...@incubator.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
