Hey everyone, JB, I fully agree with you. For clarity and consistency, we should point to the docs. The starting point for the roles can be as simple as this <https://github.com/apache/iceberg/pull/10804>. For the release manager, I clarified the how-to-release page <https://github.com/apache/iceberg/pull/10787>, which includes feedback on a recent PR <https://github.com/apache/iceberg/pull/10787>.
Let me know what you think. Kind regards, Fokko Op do 25 jul 2024 om 19:56 schreef Micah Kornfield <emkornfi...@gmail.com>: > Hi J.B. and Fokko, > > I like the idea of the state diagram, but it feels to me that there are a >> lot of states in there. Maybe we can take the Airflow AIP states as a >> start >> <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals> >> ? > > I'll take a closer look but based on the description this seems like a > good place to start. > > >> I think it would also be great to make a clear problem statement. I >> think the intention is very valuable but I don't see exactly what >> problem we are trying to solve here. > > > I think we are trying to aim/solve for the following goals/problems: > 1. To some extent there has been an unfortunate loss of trust in the > community. Providing clarity on how day to day business gets done helps > assure people with concerns that there are reasonable processes in place > that don't favor any particular corporate interest. > 2. As the Iceberg community grows what used to be a set of norms are not > necessarily well socialized among new members, having a little more > documentation on expectations on what it takes to get contributions > accepted in the project helps set expectations and empowers committers to > move things forward, hopefully increasing velocity. > 3. For larger changes, making it clearer on how to get a change through > and providing some set of norms that hopefully allows features to avoid > scope creep/analysis paralysis so the project can continue to be improved > without undo friction > > Thanks for the feedback. > > Micah > > On Thu, Jul 25, 2024 at 5:01 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> Hi, >> >> I think 1, 2 and 4 are directly up to the project. I shared what other >> ASF projects are doing (I'm thinking about Apache Beam, or Apache >> Camel Karaf), it can help to see what we can do (updating .asf.yml for >> instance). >> >> Imho, 3 & 5 should be aligned with the foundation guidelines. I don't >> see strong reasons to deviate here. >> >> I think it would also be great to make a clear problem statement. I >> think the intention is very valuable but I don't see exactly what >> problem we are trying to solve here. >> >> Regards >> JB >> >> On Tue, Jul 23, 2024 at 7:58 AM Micah Kornfield <emkornfi...@gmail.com> >> wrote: >> > >> > My 2 cents on this topic. I think we are getting bogged down in >> relatively minor details/bureaucratic points. This is a reiteration of a >> previous recommendation on the topic, but in the interest of making >> progress here, I'd propose let's break this conversation down and focus on >> incremental definitions/proposals. >> > >> > I've added a potential breakdown below. This is roughly ordered from >> codifying day to day business, and extends to longer term concerns. I >> would aim for minimalism on each of these and add additional guidance when >> we need it. It is important to keep in mind that most decisions made on a >> day to day basis are fairly easily reversible and more process/formalism >> can always be added if some part of the process has bugs in it. >> > >> > 1. Requirements to merge a PR (1 committer approval + X amount of time >> to allow others to review?) >> > 2. Requirements major projects/features including spec changes >> > 3. Define roles/responsibilities and provide guidance on how members >> can grow into those roles. (Cover release manager, comitter, PMC member, >> community member, with links as appropriate). >> > 4. Create guidelines on starting new revision to the specification >> (i.e. when will the table V3 spec be considered closed?). >> > 5. People issues (if the community thinks this is still necessary). >> > >> > Thanks, >> > Micah >> > >> > On Mon, Jul 22, 2024 at 10:05 AM Jack Ye <yezhao...@gmail.com> wrote: >> >> >> >> Just to follow up on the other topics, here are my comments, mainly to >> reconcile with what have been discussed in different threads, which could >> help formulating these multiple-choice questions: >> >> >> >> > What should the minimum voting period be? >> >> >> >> Do we decide a minimum voting period for all topics, or do we decide a >> different period for each topic? So far it seems like the ASF convention >> has different minimum periods defined, like 3 days for release [1], 7 days >> for new committers/PMC members [2]. >> >> >> >> > I'd like to include a couple sentences about the different hats at >> Apache and that votes should be for the benefit of the project and not our >> employers. >> >> > I'd like to propose that we include text to formally include censor >> and potential removal for disclosing sensitive information from the private >> list. >> >> >> >> Personally I am definitely +1 for this, but going back to the topic of >> Iceberg vs ASF bylaws/guidelines, I feel ideally this should also be a part >> of ASF general bylaws/guidelines that Iceberg simply just references. >> >> >> >> > Requirements for each topic >> >> >> >> I think this is also missing code modification and the improvement >> proposal votes. My impression so far after discussions in the initial >> bylaws doc [3] is that ASF definition of "code modification" is closer to a >> full design process and uses consensus approval. But in reality "merging >> PR" is a much more lightweight "vote" that typically requires just 1 >> committer approval of the PR. On the other hand, the Iceberg improvement >> proposal process is currently described to use the code modification >> consensus approval vote [4], but there are other options like 2/3 majority >> and majority vote that were proposed. >> >> >> >> > consensus, lazy consensus, lazy majority, lazy 2/3's >> >> >> >> In the initial bylaws doc [3] it was pointed out that the definition >> of "lazy consensus" is different in different documents. In general the >> definition is "no -1", but there is also the definition [5] of "at least 1 >> +1, no -1". I ended up giving it a different name of "minimum consensus", >> and actually this has been the model used for merging pull requests. We >> might want to clarify that part first before voting for these options. >> >> >> >> [1] https://www.apache.org/legal/release-policy.html#release-approval >> >> [2] https://community.apache.org/newcommitter.html#discussion >> >> [3] >> https://docs.google.com/document/d/1S3igb5NqSlYE3dq_qRsP3X2gwhe54fx-Sxq5hqyOe6I/edit >> >> [4] https://iceberg.apache.org/contribute/#how-are-proposals-adopted >> >> [5] https://orc.apache.org/develop/bylaws/ >> >> >> >> -Jack >> >> >> >> On Fri, Jul 19, 2024 at 2:29 PM Jack Ye <yezhao...@gmail.com> wrote: >> >>> >> >>> > specifically the discussion of the standard roles >> >>> >> >>> Yes, there are also other places with different definitions. For >> example the default project guideline [1] has additional description of the >> PMC member and chair responsibilities. There are a few other places like >> ASF glossary [2] where these terms are defined, I cannot recall on the top >> of my head, but I can try to dig those up later. >> >>> >> >>> > putting together a committer requirements/guidelines doc >> >>> >> >>> +1. For some context, the committer guideline discussion came from >> both some initial feedback on devlist, as well as comments in the roles and >> responsibilities section. >> >>> >> >>> Because I initially used the definition of Hadoop bylaws [3] for >> committers, which writes: "The project’s Committers are responsible for the >> project’s technical management." There were then multiple people pointing >> out that committers are not necessarily technical. If we look into the >> current definition in the link Owen provided: "A committer is a developer >> who has write access to the code repository and has a signed CLA on file. >> They have an apache.org mail address. Not needing to depend on other >> people to make patches to the code or documentation, they are actually >> making short-term decisions for the project." And in ASF Glossarsy: "An >> individual who has the privilege to directly commit changes to an Apache >> codebase (commit access)." >> >>> >> >>> Indeed it does not have a requirement for committers to be technical, >> but it is arguably still centered around the right to commit code and merge >> patches. And that sparked all the discussions of what exactly a committer >> means, what are the criteria, do we limit committer rights to commit to >> just sub-project, what about people that do not contribute code, etc. I >> don't know if there could be a better definition though, maybe we could >> discuss that in a separate thread that is visible to more Apache members >> and people in other projects. >> >>> >> >>> I later raised the thread >> https://lists.apache.org/thread/4fykr8316cw9lhvgq168tx5yj9zy7gc3 where >> you can find more details about the discussions in the thread as well as in >> the linked Google doc for committer guideline. Topics like minimum time >> requirement and recency requirements were discussed there. Personally, I >> included all those aspects in the draft because my main goal is to have an >> open discussion about these topics so that we can potentially reach some >> consensus, and eventually form a clear guideline. It would be nice if we >> can do that as a part of this process and also moderated in the same way. >> >>> >> >>> -Jack >> >>> >> >>> [1] >> https://cwiki.apache.org/confluence/display/INCUBATOR/Default+Project+Guidelines >> >>> [2] https://www.apache.org/foundation/glossary >> >>> [3] https://hadoop.apache.org/bylaws.html >> >>> >> >>> >> >>> On Fri, Jul 19, 2024 at 11:54 AM Ryan Blue >> <b...@databricks.com.invalid> wrote: >> >>>> >> >>>> Another thing that has come up is putting together a committer >> requirements/guidelines doc. I think it would be great to have a discussion >> about how we want to do that. Specifically, I'm against adding requirements >> for new committers (such as time-based minimums or recency requirements) >> that exclude people from consideration. I think it would be helpful to >> clarify the path to becoming a committer, though. To me, it's about >> building trust and that trust is recognized by a PMC vote. >> >>>> >> >>>> Ryan >> >>>> >> >>>> On Fri, Jul 19, 2024 at 11:00 AM Owen O'Malley < >> owen.omal...@gmail.com> wrote: >> >>>>> >> >>>>> I meant specifically the discussion of the standard roles (eg. >> users, committers, pmc, pmc chair) that are well covered in >> https://www.apache.org/foundation/how-it-works/#roles >> >>>>> >> >>>>> .. Owen >> >>>>> >> >>>>> On Fri, Jul 19, 2024 at 10:43 AM Jack Ye <yezhao...@gmail.com> >> wrote: >> >>>>>> >> >>>>>> Thank you Owen for moving this forward, we heard you were sick, >> hope you are fully recovered now! >> >>>>>> >> >>>>>> One point regarding "referring to the Apache documentation": I am >> totally for that, but during the initial investigation, I found out that >> the Apache documentations are scattered around, and also have conflicting >> information. >> >>>>>> >> >>>>>> For example, regarding a "vote for committer or PMC member": >> >>>>>> - this new committer doc [1] writes that "A positive result is >> achieved by Consensus Approval: at least 3 +1 votes and no vetoes." >> >>>>>> - the Apache voting process doc [2] writes that "Votes on >> procedural issues follow the common format of majority rule unless >> otherwise stated.", and when consulting a few Apache members, most of them >> consider voting for committer or PMC member a procedural issue. >> >>>>>> >> >>>>>> Similar situations were found for other topics like description of >> roles and responsibilities, code modification, etc. >> >>>>>> >> >>>>>> I think it is a great chance for ASF in general to consolidate >> these information, especially for matters that have common guidelines in >> ASF that should be adhered to by all projects. With that, we can figure out >> what to put in the Iceberg specific bylaws, either to directly refer to ASF >> official information, or to add additional information and guidelines. >> >>>>>> >> >>>>>> Regarding sub-projects, the main reason I proposed it at the >> beginning was to allow a proper definition of release manager >> responsibility, since each sub-project is released independently. It was >> not intended to be tied to committer responsibilities. >> >>>>>> >> >>>>>> Best, >> >>>>>> Jack Ye >> >>>>>> >> >>>>>> [1] https://community.apache.org/newcommitter.html >> >>>>>> [2] https://www.apache.org/foundation/voting >> >>>>>> >> >>>>>> On Fri, Jul 19, 2024 at 10:22 AM Owen O'Malley < >> owen.omal...@gmail.com> wrote: >> >>>>>>> >> >>>>>>> Everyone is welcome to vote. The Iceberg PMC will have the only >> binding votes. >> >>>>>>> >> >>>>>>> .. Owen >> >>>>>>> >> >>>>>>> On Jul 19, 2024, at 10:19, Wing Yew Poon >> <wyp...@cloudera.com.invalid> wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> Hi Owen, >> >>>>>>> Thanks for doing this. >> >>>>>>> Once you have the questions and choices, who gets to vote on them? >> >>>>>>> - Wing Yew >> >>>>>>> >> >>>>>>> >> >>>>>>> On Fri, Jul 19, 2024 at 10:07 AM Owen O'Malley < >> owen.omal...@gmail.com> wrote: >> >>>>>>>> >> >>>>>>>> All, >> >>>>>>>> Sorry for the long pause on bylaws discussion. It was a >> result of wanting to avoid the long US holiday week (July 4th) and my >> procrastination, which was furthered by a side conversation that asked me >> to consider how to move forward in an Apache way. >> >>>>>>>> I'd like to thank Jack for moving this to this point. One >> concern that I had was there were lots of discussions and decisions that >> were being made off of our email lists, which isn't the way that Apache >> should work. >> >>>>>>>> For finishing this off, I'd like to come up with a set of >> questions that should be answered by multiple choice questions and then use >> single transferable vote (STV) to resolve them. STV just means that each >> person lists their choices in a ranked order with a formal way to resolve >> how the votes work. >> >>>>>>>> The questions that I have heard so far are: >> >>>>>>>> >> >>>>>>>> Should the PMC chair be term-limited and if so, what is the >> period? In my experience, this isn't necessary in most projects and is >> often ignored. In Hadoop, Chris Douglas was a great chair and held it for 5 >> years in spite of the 1 year limit. >> >>>>>>>> >> >>>>>>>> No term limit >> >>>>>>>> 1 year >> >>>>>>>> 2 year >> >>>>>>>> >> >>>>>>>> What should the minimum voting period be? I'd suggest 3 days is >> far better as long as it isn't abused by holding important votes over >> holiday weekends. >> >>>>>>>> >> >>>>>>>> 3 days (72 hours) >> >>>>>>>> 7 days >> >>>>>>>> >> >>>>>>>> Should we keep the section on roles or just reference the Apache >> documentation. I'd suggest that we reference the Apache documentation. >> >>>>>>>> I'd like to include a couple sentences about the different hats >> at Apache and that votes should be for the benefit of the project and not >> our employers. >> >>>>>>>> I'd like to propose that we include text to formally include >> censor and potential removal for disclosing sensitive information from the >> private list. >> >>>>>>>> I'd like to propose branch committers. It has helped Hadoop a >> lot to enable people to work on development branches for large features >> before they are given general committership. It is better to have the >> branch work done at Apache and be visible than having large branches come >> in late in the project. >> >>>>>>>> Requirements for each topic (each could be consensus, lazy >> consensus, lazy majority, lazy 2/3's) >> >>>>>>>> >> >>>>>>>> Add committer >> >>>>>>>> Remove committer >> >>>>>>>> Add PMC >> >>>>>>>> Remove PMC >> >>>>>>>> Accept design proposal >> >>>>>>>> Add subproject >> >>>>>>>> Remove subproject >> >>>>>>>> Release (can't be lazy consensus) >> >>>>>>>> Modifying bylaws >> >>>>>>>> >> >>>>>>>> Thoughts? Missing questions? >> >>>>>>>> >> >>>>>>>> .. Owen >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Ryan Blue >> >>>> Databricks >> >
