Thanks, Fokko! I agree that those changes look like good additions to clarify the release process and roles.
On Mon, Jul 29, 2024 at 4:39 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Fokko > > Thanks for these changes, it looks great to me ! > > Regards > JB > > On Mon, Jul 29, 2024 at 12:40 PM Fokko Driesprong <fo...@apache.org> > wrote: > > > > Resending with correct links :p > > > > Hey everyone, > > > > JB, I fully agree with you. For clarity and consistency, we should point > to the docs. > > The starting point for the roles can be as simple as this. For the > release manager, > > I clarified the how-to-release page, which includes feedback on a recent > PR. > > > > Let me know what you think. > > > > Kind regards, > > Fokko > > > > Op ma 29 jul 2024 om 12:27 schreef Fokko Driesprong <fo...@apache.org>: > >> > >> Hey everyone, > >> > >> JB, I fully agree with you. For clarity and consistency, we should > point to the docs. > >> The starting point for the roles can be as simple as this. For the > release manager, > >> I clarified the how-to-release page, which includes feedback on a > recent PR. > >> > >> Let me know what you think. > >> > >> Kind regards, > >> Fokko > >> > >> Op do 25 jul 2024 om 19:56 schreef Micah Kornfield < > emkornfi...@gmail.com>: > >>> > >>> Hi J.B. and Fokko, > >>> > >>>> I like the idea of the state diagram, but it feels to me that there > are a lot of states in there. Maybe we can take the Airflow AIP states as a > start? > >>> > >>> I'll take a closer look but based on the description this seems like a > good place to start. > >>> > >>>> > >>>> I think it would also be great to make a clear problem statement. I > >>>> think the intention is very valuable but I don't see exactly what > >>>> problem we are trying to solve here. > >>> > >>> > >>> I think we are trying to aim/solve for the following goals/problems: > >>> 1. To some extent there has been an unfortunate loss of trust in the > community. Providing clarity on how day to day business gets done helps > assure people with concerns that there are reasonable processes in place > that don't favor any particular corporate interest. > >>> 2. As the Iceberg community grows what used to be a set of norms are > not necessarily well socialized among new members, having a little more > documentation on expectations on what it takes to get contributions > accepted in the project helps set expectations and empowers committers to > move things forward, hopefully increasing velocity. > >>> 3. For larger changes, making it clearer on how to get a change > through and providing some set of norms that hopefully allows features to > avoid scope creep/analysis paralysis so the project can continue to be > improved without undo friction > >>> > >>> Thanks for the feedback. > >>> > >>> Micah > >>> > >>> On Thu, Jul 25, 2024 at 5:01 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >>>> > >>>> Hi, > >>>> > >>>> I think 1, 2 and 4 are directly up to the project. I shared what other > >>>> ASF projects are doing (I'm thinking about Apache Beam, or Apache > >>>> Camel Karaf), it can help to see what we can do (updating .asf.yml for > >>>> instance). > >>>> > >>>> Imho, 3 & 5 should be aligned with the foundation guidelines. I don't > >>>> see strong reasons to deviate here. > >>>> > >>>> I think it would also be great to make a clear problem statement. I > >>>> think the intention is very valuable but I don't see exactly what > >>>> problem we are trying to solve here. > >>>> > >>>> Regards > >>>> JB > >>>> > >>>> On Tue, Jul 23, 2024 at 7:58 AM Micah Kornfield < > emkornfi...@gmail.com> wrote: > >>>> > > >>>> > My 2 cents on this topic. I think we are getting bogged down in > relatively minor details/bureaucratic points. This is a reiteration of a > previous recommendation on the topic, but in the interest of making > progress here, I'd propose let's break this conversation down and focus on > incremental definitions/proposals. > >>>> > > >>>> > I've added a potential breakdown below. This is roughly ordered > from codifying day to day business, and extends to longer term concerns. I > would aim for minimalism on each of these and add additional guidance when > we need it. It is important to keep in mind that most decisions made on a > day to day basis are fairly easily reversible and more process/formalism > can always be added if some part of the process has bugs in it. > >>>> > > >>>> > 1. Requirements to merge a PR (1 committer approval + X amount of > time to allow others to review?) > >>>> > 2. Requirements major projects/features including spec changes > >>>> > 3. Define roles/responsibilities and provide guidance on how > members can grow into those roles. (Cover release manager, comitter, PMC > member, community member, with links as appropriate). > >>>> > 4. Create guidelines on starting new revision to the specification > (i.e. when will the table V3 spec be considered closed?). > >>>> > 5. People issues (if the community thinks this is still necessary). > >>>> > > >>>> > Thanks, > >>>> > Micah > >>>> > > >>>> > On Mon, Jul 22, 2024 at 10:05 AM Jack Ye <yezhao...@gmail.com> > wrote: > >>>> >> > >>>> >> Just to follow up on the other topics, here are my comments, > mainly to reconcile with what have been discussed in different threads, > which could help formulating these multiple-choice questions: > >>>> >> > >>>> >> > What should the minimum voting period be? > >>>> >> > >>>> >> Do we decide a minimum voting period for all topics, or do we > decide a different period for each topic? So far it seems like the ASF > convention has different minimum periods defined, like 3 days for release > [1], 7 days for new committers/PMC members [2]. > >>>> >> > >>>> >> > I'd like to include a couple sentences about the different hats > at Apache and that votes should be for the benefit of the project and not > our employers. > >>>> >> > I'd like to propose that we include text to formally include > censor and potential removal for disclosing sensitive information from the > private list. > >>>> >> > >>>> >> Personally I am definitely +1 for this, but going back to the > topic of Iceberg vs ASF bylaws/guidelines, I feel ideally this should also > be a part of ASF general bylaws/guidelines that Iceberg simply just > references. > >>>> >> > >>>> >> > Requirements for each topic > >>>> >> > >>>> >> I think this is also missing code modification and the improvement > proposal votes. My impression so far after discussions in the initial > bylaws doc [3] is that ASF definition of "code modification" is closer to a > full design process and uses consensus approval. But in reality "merging > PR" is a much more lightweight "vote" that typically requires just 1 > committer approval of the PR. On the other hand, the Iceberg improvement > proposal process is currently described to use the code modification > consensus approval vote [4], but there are other options like 2/3 majority > and majority vote that were proposed. > >>>> >> > >>>> >> > consensus, lazy consensus, lazy majority, lazy 2/3's > >>>> >> > >>>> >> In the initial bylaws doc [3] it was pointed out that the > definition of "lazy consensus" is different in different documents. In > general the definition is "no -1", but there is also the definition [5] of > "at least 1 +1, no -1". I ended up giving it a different name of "minimum > consensus", and actually this has been the model used for merging pull > requests. We might want to clarify that part first before voting for these > options. > >>>> >> > >>>> >> [1] > https://www.apache.org/legal/release-policy.html#release-approval > >>>> >> [2] https://community.apache.org/newcommitter.html#discussion > >>>> >> [3] > https://docs.google.com/document/d/1S3igb5NqSlYE3dq_qRsP3X2gwhe54fx-Sxq5hqyOe6I/edit > >>>> >> [4] > https://iceberg.apache.org/contribute/#how-are-proposals-adopted > >>>> >> [5] https://orc.apache.org/develop/bylaws/ > >>>> >> > >>>> >> -Jack > >>>> >> > >>>> >> On Fri, Jul 19, 2024 at 2:29 PM Jack Ye <yezhao...@gmail.com> > wrote: > >>>> >>> > >>>> >>> > specifically the discussion of the standard roles > >>>> >>> > >>>> >>> Yes, there are also other places with different definitions. For > example the default project guideline [1] has additional description of the > PMC member and chair responsibilities. There are a few other places like > ASF glossary [2] where these terms are defined, I cannot recall on the top > of my head, but I can try to dig those up later. > >>>> >>> > >>>> >>> > putting together a committer requirements/guidelines doc > >>>> >>> > >>>> >>> +1. For some context, the committer guideline discussion came > from both some initial feedback on devlist, as well as comments in the > roles and responsibilities section. > >>>> >>> > >>>> >>> Because I initially used the definition of Hadoop bylaws [3] for > committers, which writes: "The project’s Committers are responsible for the > project’s technical management." There were then multiple people pointing > out that committers are not necessarily technical. If we look into the > current definition in the link Owen provided: "A committer is a developer > who has write access to the code repository and has a signed CLA on file. > They have an apache.org mail address. Not needing to depend on other > people to make patches to the code or documentation, they are actually > making short-term decisions for the project." And in ASF Glossarsy: "An > individual who has the privilege to directly commit changes to an Apache > codebase (commit access)." > >>>> >>> > >>>> >>> Indeed it does not have a requirement for committers to be > technical, but it is arguably still centered around the right to commit > code and merge patches. And that sparked all the discussions of what > exactly a committer means, what are the criteria, do we limit committer > rights to commit to just sub-project, what about people that do not > contribute code, etc. I don't know if there could be a better definition > though, maybe we could discuss that in a separate thread that is visible to > more Apache members and people in other projects. > >>>> >>> > >>>> >>> I later raised the thread > https://lists.apache.org/thread/4fykr8316cw9lhvgq168tx5yj9zy7gc3 where > you can find more details about the discussions in the thread as well as in > the linked Google doc for committer guideline. Topics like minimum time > requirement and recency requirements were discussed there. Personally, I > included all those aspects in the draft because my main goal is to have an > open discussion about these topics so that we can potentially reach some > consensus, and eventually form a clear guideline. It would be nice if we > can do that as a part of this process and also moderated in the same way. > >>>> >>> > >>>> >>> -Jack > >>>> >>> > >>>> >>> [1] > https://cwiki.apache.org/confluence/display/INCUBATOR/Default+Project+Guidelines > >>>> >>> [2] https://www.apache.org/foundation/glossary > >>>> >>> [3] https://hadoop.apache.org/bylaws.html > >>>> >>> > >>>> >>> > >>>> >>> On Fri, Jul 19, 2024 at 11:54 AM Ryan Blue > <b...@databricks.com.invalid> wrote: > >>>> >>>> > >>>> >>>> Another thing that has come up is putting together a committer > requirements/guidelines doc. I think it would be great to have a discussion > about how we want to do that. Specifically, I'm against adding requirements > for new committers (such as time-based minimums or recency requirements) > that exclude people from consideration. I think it would be helpful to > clarify the path to becoming a committer, though. To me, it's about > building trust and that trust is recognized by a PMC vote. > >>>> >>>> > >>>> >>>> Ryan > >>>> >>>> > >>>> >>>> On Fri, Jul 19, 2024 at 11:00 AM Owen O'Malley < > owen.omal...@gmail.com> wrote: > >>>> >>>>> > >>>> >>>>> I meant specifically the discussion of the standard roles (eg. > users, committers, pmc, pmc chair) that are well covered in > https://www.apache.org/foundation/how-it-works/#roles > >>>> >>>>> > >>>> >>>>> .. Owen > >>>> >>>>> > >>>> >>>>> On Fri, Jul 19, 2024 at 10:43 AM Jack Ye <yezhao...@gmail.com> > wrote: > >>>> >>>>>> > >>>> >>>>>> Thank you Owen for moving this forward, we heard you were > sick, hope you are fully recovered now! > >>>> >>>>>> > >>>> >>>>>> One point regarding "referring to the Apache documentation": I > am totally for that, but during the initial investigation, I found out that > the Apache documentations are scattered around, and also have conflicting > information. > >>>> >>>>>> > >>>> >>>>>> For example, regarding a "vote for committer or PMC member": > >>>> >>>>>> - this new committer doc [1] writes that "A positive result is > achieved by Consensus Approval: at least 3 +1 votes and no vetoes." > >>>> >>>>>> - the Apache voting process doc [2] writes that "Votes on > procedural issues follow the common format of majority rule unless > otherwise stated.", and when consulting a few Apache members, most of them > consider voting for committer or PMC member a procedural issue. > >>>> >>>>>> > >>>> >>>>>> Similar situations were found for other topics like > description of roles and responsibilities, code modification, etc. > >>>> >>>>>> > >>>> >>>>>> I think it is a great chance for ASF in general to consolidate > these information, especially for matters that have common guidelines in > ASF that should be adhered to by all projects. With that, we can figure out > what to put in the Iceberg specific bylaws, either to directly refer to ASF > official information, or to add additional information and guidelines. > >>>> >>>>>> > >>>> >>>>>> Regarding sub-projects, the main reason I proposed it at the > beginning was to allow a proper definition of release manager > responsibility, since each sub-project is released independently. It was > not intended to be tied to committer responsibilities. > >>>> >>>>>> > >>>> >>>>>> Best, > >>>> >>>>>> Jack Ye > >>>> >>>>>> > >>>> >>>>>> [1] https://community.apache.org/newcommitter.html > >>>> >>>>>> [2] https://www.apache.org/foundation/voting > >>>> >>>>>> > >>>> >>>>>> On Fri, Jul 19, 2024 at 10:22 AM Owen O'Malley < > owen.omal...@gmail.com> wrote: > >>>> >>>>>>> > >>>> >>>>>>> Everyone is welcome to vote. The Iceberg PMC will have the > only binding votes. > >>>> >>>>>>> > >>>> >>>>>>> .. Owen > >>>> >>>>>>> > >>>> >>>>>>> On Jul 19, 2024, at 10:19, Wing Yew Poon > <wyp...@cloudera.com.invalid> wrote: > >>>> >>>>>>> > >>>> >>>>>>> > >>>> >>>>>>> Hi Owen, > >>>> >>>>>>> Thanks for doing this. > >>>> >>>>>>> Once you have the questions and choices, who gets to vote on > them? > >>>> >>>>>>> - Wing Yew > >>>> >>>>>>> > >>>> >>>>>>> > >>>> >>>>>>> On Fri, Jul 19, 2024 at 10:07 AM Owen O'Malley < > owen.omal...@gmail.com> wrote: > >>>> >>>>>>>> > >>>> >>>>>>>> All, > >>>> >>>>>>>> Sorry for the long pause on bylaws discussion. It was a > result of wanting to avoid the long US holiday week (July 4th) and my > procrastination, which was furthered by a side conversation that asked me > to consider how to move forward in an Apache way. > >>>> >>>>>>>> I'd like to thank Jack for moving this to this point. One > concern that I had was there were lots of discussions and decisions that > were being made off of our email lists, which isn't the way that Apache > should work. > >>>> >>>>>>>> For finishing this off, I'd like to come up with a set of > questions that should be answered by multiple choice questions and then use > single transferable vote (STV) to resolve them. STV just means that each > person lists their choices in a ranked order with a formal way to resolve > how the votes work. > >>>> >>>>>>>> The questions that I have heard so far are: > >>>> >>>>>>>> > >>>> >>>>>>>> Should the PMC chair be term-limited and if so, what is the > period? In my experience, this isn't necessary in most projects and is > often ignored. In Hadoop, Chris Douglas was a great chair and held it for 5 > years in spite of the 1 year limit. > >>>> >>>>>>>> > >>>> >>>>>>>> No term limit > >>>> >>>>>>>> 1 year > >>>> >>>>>>>> 2 year > >>>> >>>>>>>> > >>>> >>>>>>>> What should the minimum voting period be? I'd suggest 3 days > is far better as long as it isn't abused by holding important votes over > holiday weekends. > >>>> >>>>>>>> > >>>> >>>>>>>> 3 days (72 hours) > >>>> >>>>>>>> 7 days > >>>> >>>>>>>> > >>>> >>>>>>>> Should we keep the section on roles or just reference the > Apache documentation. I'd suggest that we reference the Apache > documentation. > >>>> >>>>>>>> I'd like to include a couple sentences about the different > hats at Apache and that votes should be for the benefit of the project and > not our employers. > >>>> >>>>>>>> I'd like to propose that we include text to formally include > censor and potential removal for disclosing sensitive information from the > private list. > >>>> >>>>>>>> I'd like to propose branch committers. It has helped Hadoop > a lot to enable people to work on development branches for large features > before they are given general committership. It is better to have the > branch work done at Apache and be visible than having large branches come > in late in the project. > >>>> >>>>>>>> Requirements for each topic (each could be consensus, lazy > consensus, lazy majority, lazy 2/3's) > >>>> >>>>>>>> > >>>> >>>>>>>> Add committer > >>>> >>>>>>>> Remove committer > >>>> >>>>>>>> Add PMC > >>>> >>>>>>>> Remove PMC > >>>> >>>>>>>> Accept design proposal > >>>> >>>>>>>> Add subproject > >>>> >>>>>>>> Remove subproject > >>>> >>>>>>>> Release (can't be lazy consensus) > >>>> >>>>>>>> Modifying bylaws > >>>> >>>>>>>> > >>>> >>>>>>>> Thoughts? Missing questions? > >>>> >>>>>>>> > >>>> >>>>>>>> .. Owen > >>>> >>>> > >>>> >>>> > >>>> >>>> > >>>> >>>> -- > >>>> >>>> Ryan Blue > >>>> >>>> Databricks > -- Ryan Blue Databricks
