Resending with correct links :p Hey everyone,
JB, I fully agree with you. For clarity and consistency, we should point to the docs. The starting point for the roles can be as simple as this <https://github.com/apache/iceberg/pull/10804>. For the release manager, I clarified the how-to-release page <https://github.com/apache/iceberg/pull/10806>, which includes feedback on a recent PR <https://github.com/apache/iceberg/pull/10787>. Let me know what you think. Kind regards, Fokko Op ma 29 jul 2024 om 12:27 schreef Fokko Driesprong <fo...@apache.org>: > Hey everyone, > > JB, I fully agree with you. For clarity and consistency, we should point > to the docs. > The starting point for the roles can be as simple as this > <https://github.com/apache/iceberg/pull/10804>. For the release manager, > I clarified the how-to-release page > <https://github.com/apache/iceberg/pull/10787>, which includes feedback > on a recent PR <https://github.com/apache/iceberg/pull/10787>. > > Let me know what you think. > > Kind regards, > Fokko > > Op do 25 jul 2024 om 19:56 schreef Micah Kornfield <emkornfi...@gmail.com > >: > >> Hi J.B. and Fokko, >> >> I like the idea of the state diagram, but it feels to me that there are a >>> lot of states in there. Maybe we can take the Airflow AIP states as a >>> start >>> <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals> >>> ? >> >> I'll take a closer look but based on the description this seems like a >> good place to start. >> >> >>> I think it would also be great to make a clear problem statement. I >>> think the intention is very valuable but I don't see exactly what >>> problem we are trying to solve here. >> >> >> I think we are trying to aim/solve for the following goals/problems: >> 1. To some extent there has been an unfortunate loss of trust in the >> community. Providing clarity on how day to day business gets done helps >> assure people with concerns that there are reasonable processes in place >> that don't favor any particular corporate interest. >> 2. As the Iceberg community grows what used to be a set of norms are not >> necessarily well socialized among new members, having a little more >> documentation on expectations on what it takes to get contributions >> accepted in the project helps set expectations and empowers committers to >> move things forward, hopefully increasing velocity. >> 3. For larger changes, making it clearer on how to get a change through >> and providing some set of norms that hopefully allows features to avoid >> scope creep/analysis paralysis so the project can continue to be improved >> without undo friction >> >> Thanks for the feedback. >> >> Micah >> >> On Thu, Jul 25, 2024 at 5:01 AM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >> >>> Hi, >>> >>> I think 1, 2 and 4 are directly up to the project. I shared what other >>> ASF projects are doing (I'm thinking about Apache Beam, or Apache >>> Camel Karaf), it can help to see what we can do (updating .asf.yml for >>> instance). >>> >>> Imho, 3 & 5 should be aligned with the foundation guidelines. I don't >>> see strong reasons to deviate here. >>> >>> I think it would also be great to make a clear problem statement. I >>> think the intention is very valuable but I don't see exactly what >>> problem we are trying to solve here. >>> >>> Regards >>> JB >>> >>> On Tue, Jul 23, 2024 at 7:58 AM Micah Kornfield <emkornfi...@gmail.com> >>> wrote: >>> > >>> > My 2 cents on this topic. I think we are getting bogged down in >>> relatively minor details/bureaucratic points. This is a reiteration of a >>> previous recommendation on the topic, but in the interest of making >>> progress here, I'd propose let's break this conversation down and focus on >>> incremental definitions/proposals. >>> > >>> > I've added a potential breakdown below. This is roughly ordered from >>> codifying day to day business, and extends to longer term concerns. I >>> would aim for minimalism on each of these and add additional guidance when >>> we need it. It is important to keep in mind that most decisions made on a >>> day to day basis are fairly easily reversible and more process/formalism >>> can always be added if some part of the process has bugs in it. >>> > >>> > 1. Requirements to merge a PR (1 committer approval + X amount of >>> time to allow others to review?) >>> > 2. Requirements major projects/features including spec changes >>> > 3. Define roles/responsibilities and provide guidance on how members >>> can grow into those roles. (Cover release manager, comitter, PMC member, >>> community member, with links as appropriate). >>> > 4. Create guidelines on starting new revision to the specification >>> (i.e. when will the table V3 spec be considered closed?). >>> > 5. People issues (if the community thinks this is still necessary). >>> > >>> > Thanks, >>> > Micah >>> > >>> > On Mon, Jul 22, 2024 at 10:05 AM Jack Ye <yezhao...@gmail.com> wrote: >>> >> >>> >> Just to follow up on the other topics, here are my comments, mainly >>> to reconcile with what have been discussed in different threads, which >>> could help formulating these multiple-choice questions: >>> >> >>> >> > What should the minimum voting period be? >>> >> >>> >> Do we decide a minimum voting period for all topics, or do we decide >>> a different period for each topic? So far it seems like the ASF convention >>> has different minimum periods defined, like 3 days for release [1], 7 days >>> for new committers/PMC members [2]. >>> >> >>> >> > I'd like to include a couple sentences about the different hats at >>> Apache and that votes should be for the benefit of the project and not our >>> employers. >>> >> > I'd like to propose that we include text to formally include censor >>> and potential removal for disclosing sensitive information from the private >>> list. >>> >> >>> >> Personally I am definitely +1 for this, but going back to the topic >>> of Iceberg vs ASF bylaws/guidelines, I feel ideally this should also be a >>> part of ASF general bylaws/guidelines that Iceberg simply just references. >>> >> >>> >> > Requirements for each topic >>> >> >>> >> I think this is also missing code modification and the improvement >>> proposal votes. My impression so far after discussions in the initial >>> bylaws doc [3] is that ASF definition of "code modification" is closer to a >>> full design process and uses consensus approval. But in reality "merging >>> PR" is a much more lightweight "vote" that typically requires just 1 >>> committer approval of the PR. On the other hand, the Iceberg improvement >>> proposal process is currently described to use the code modification >>> consensus approval vote [4], but there are other options like 2/3 majority >>> and majority vote that were proposed. >>> >> >>> >> > consensus, lazy consensus, lazy majority, lazy 2/3's >>> >> >>> >> In the initial bylaws doc [3] it was pointed out that the definition >>> of "lazy consensus" is different in different documents. In general the >>> definition is "no -1", but there is also the definition [5] of "at least 1 >>> +1, no -1". I ended up giving it a different name of "minimum consensus", >>> and actually this has been the model used for merging pull requests. We >>> might want to clarify that part first before voting for these options. >>> >> >>> >> [1] https://www.apache.org/legal/release-policy.html#release-approval >>> >> [2] https://community.apache.org/newcommitter.html#discussion >>> >> [3] >>> https://docs.google.com/document/d/1S3igb5NqSlYE3dq_qRsP3X2gwhe54fx-Sxq5hqyOe6I/edit >>> >> [4] https://iceberg.apache.org/contribute/#how-are-proposals-adopted >>> >> [5] https://orc.apache.org/develop/bylaws/ >>> >> >>> >> -Jack >>> >> >>> >> On Fri, Jul 19, 2024 at 2:29 PM Jack Ye <yezhao...@gmail.com> wrote: >>> >>> >>> >>> > specifically the discussion of the standard roles >>> >>> >>> >>> Yes, there are also other places with different definitions. For >>> example the default project guideline [1] has additional description of the >>> PMC member and chair responsibilities. There are a few other places like >>> ASF glossary [2] where these terms are defined, I cannot recall on the top >>> of my head, but I can try to dig those up later. >>> >>> >>> >>> > putting together a committer requirements/guidelines doc >>> >>> >>> >>> +1. For some context, the committer guideline discussion came from >>> both some initial feedback on devlist, as well as comments in the roles and >>> responsibilities section. >>> >>> >>> >>> Because I initially used the definition of Hadoop bylaws [3] for >>> committers, which writes: "The project’s Committers are responsible for the >>> project’s technical management." There were then multiple people pointing >>> out that committers are not necessarily technical. If we look into the >>> current definition in the link Owen provided: "A committer is a developer >>> who has write access to the code repository and has a signed CLA on file. >>> They have an apache.org mail address. Not needing to depend on other >>> people to make patches to the code or documentation, they are actually >>> making short-term decisions for the project." And in ASF Glossarsy: "An >>> individual who has the privilege to directly commit changes to an Apache >>> codebase (commit access)." >>> >>> >>> >>> Indeed it does not have a requirement for committers to be >>> technical, but it is arguably still centered around the right to commit >>> code and merge patches. And that sparked all the discussions of what >>> exactly a committer means, what are the criteria, do we limit committer >>> rights to commit to just sub-project, what about people that do not >>> contribute code, etc. I don't know if there could be a better definition >>> though, maybe we could discuss that in a separate thread that is visible to >>> more Apache members and people in other projects. >>> >>> >>> >>> I later raised the thread >>> https://lists.apache.org/thread/4fykr8316cw9lhvgq168tx5yj9zy7gc3 where >>> you can find more details about the discussions in the thread as well as in >>> the linked Google doc for committer guideline. Topics like minimum time >>> requirement and recency requirements were discussed there. Personally, I >>> included all those aspects in the draft because my main goal is to have an >>> open discussion about these topics so that we can potentially reach some >>> consensus, and eventually form a clear guideline. It would be nice if we >>> can do that as a part of this process and also moderated in the same way. >>> >>> >>> >>> -Jack >>> >>> >>> >>> [1] >>> https://cwiki.apache.org/confluence/display/INCUBATOR/Default+Project+Guidelines >>> >>> [2] https://www.apache.org/foundation/glossary >>> >>> [3] https://hadoop.apache.org/bylaws.html >>> >>> >>> >>> >>> >>> On Fri, Jul 19, 2024 at 11:54 AM Ryan Blue >>> <b...@databricks.com.invalid> wrote: >>> >>>> >>> >>>> Another thing that has come up is putting together a committer >>> requirements/guidelines doc. I think it would be great to have a discussion >>> about how we want to do that. Specifically, I'm against adding requirements >>> for new committers (such as time-based minimums or recency requirements) >>> that exclude people from consideration. I think it would be helpful to >>> clarify the path to becoming a committer, though. To me, it's about >>> building trust and that trust is recognized by a PMC vote. >>> >>>> >>> >>>> Ryan >>> >>>> >>> >>>> On Fri, Jul 19, 2024 at 11:00 AM Owen O'Malley < >>> owen.omal...@gmail.com> wrote: >>> >>>>> >>> >>>>> I meant specifically the discussion of the standard roles (eg. >>> users, committers, pmc, pmc chair) that are well covered in >>> https://www.apache.org/foundation/how-it-works/#roles >>> >>>>> >>> >>>>> .. Owen >>> >>>>> >>> >>>>> On Fri, Jul 19, 2024 at 10:43 AM Jack Ye <yezhao...@gmail.com> >>> wrote: >>> >>>>>> >>> >>>>>> Thank you Owen for moving this forward, we heard you were sick, >>> hope you are fully recovered now! >>> >>>>>> >>> >>>>>> One point regarding "referring to the Apache documentation": I am >>> totally for that, but during the initial investigation, I found out that >>> the Apache documentations are scattered around, and also have conflicting >>> information. >>> >>>>>> >>> >>>>>> For example, regarding a "vote for committer or PMC member": >>> >>>>>> - this new committer doc [1] writes that "A positive result is >>> achieved by Consensus Approval: at least 3 +1 votes and no vetoes." >>> >>>>>> - the Apache voting process doc [2] writes that "Votes on >>> procedural issues follow the common format of majority rule unless >>> otherwise stated.", and when consulting a few Apache members, most of them >>> consider voting for committer or PMC member a procedural issue. >>> >>>>>> >>> >>>>>> Similar situations were found for other topics like description >>> of roles and responsibilities, code modification, etc. >>> >>>>>> >>> >>>>>> I think it is a great chance for ASF in general to consolidate >>> these information, especially for matters that have common guidelines in >>> ASF that should be adhered to by all projects. With that, we can figure out >>> what to put in the Iceberg specific bylaws, either to directly refer to ASF >>> official information, or to add additional information and guidelines. >>> >>>>>> >>> >>>>>> Regarding sub-projects, the main reason I proposed it at the >>> beginning was to allow a proper definition of release manager >>> responsibility, since each sub-project is released independently. It was >>> not intended to be tied to committer responsibilities. >>> >>>>>> >>> >>>>>> Best, >>> >>>>>> Jack Ye >>> >>>>>> >>> >>>>>> [1] https://community.apache.org/newcommitter.html >>> >>>>>> [2] https://www.apache.org/foundation/voting >>> >>>>>> >>> >>>>>> On Fri, Jul 19, 2024 at 10:22 AM Owen O'Malley < >>> owen.omal...@gmail.com> wrote: >>> >>>>>>> >>> >>>>>>> Everyone is welcome to vote. The Iceberg PMC will have the only >>> binding votes. >>> >>>>>>> >>> >>>>>>> .. Owen >>> >>>>>>> >>> >>>>>>> On Jul 19, 2024, at 10:19, Wing Yew Poon >>> <wyp...@cloudera.com.invalid> wrote: >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> Hi Owen, >>> >>>>>>> Thanks for doing this. >>> >>>>>>> Once you have the questions and choices, who gets to vote on >>> them? >>> >>>>>>> - Wing Yew >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> On Fri, Jul 19, 2024 at 10:07 AM Owen O'Malley < >>> owen.omal...@gmail.com> wrote: >>> >>>>>>>> >>> >>>>>>>> All, >>> >>>>>>>> Sorry for the long pause on bylaws discussion. It was a >>> result of wanting to avoid the long US holiday week (July 4th) and my >>> procrastination, which was furthered by a side conversation that asked me >>> to consider how to move forward in an Apache way. >>> >>>>>>>> I'd like to thank Jack for moving this to this point. One >>> concern that I had was there were lots of discussions and decisions that >>> were being made off of our email lists, which isn't the way that Apache >>> should work. >>> >>>>>>>> For finishing this off, I'd like to come up with a set of >>> questions that should be answered by multiple choice questions and then use >>> single transferable vote (STV) to resolve them. STV just means that each >>> person lists their choices in a ranked order with a formal way to resolve >>> how the votes work. >>> >>>>>>>> The questions that I have heard so far are: >>> >>>>>>>> >>> >>>>>>>> Should the PMC chair be term-limited and if so, what is the >>> period? In my experience, this isn't necessary in most projects and is >>> often ignored. In Hadoop, Chris Douglas was a great chair and held it for 5 >>> years in spite of the 1 year limit. >>> >>>>>>>> >>> >>>>>>>> No term limit >>> >>>>>>>> 1 year >>> >>>>>>>> 2 year >>> >>>>>>>> >>> >>>>>>>> What should the minimum voting period be? I'd suggest 3 days is >>> far better as long as it isn't abused by holding important votes over >>> holiday weekends. >>> >>>>>>>> >>> >>>>>>>> 3 days (72 hours) >>> >>>>>>>> 7 days >>> >>>>>>>> >>> >>>>>>>> Should we keep the section on roles or just reference the >>> Apache documentation. I'd suggest that we reference the Apache >>> documentation. >>> >>>>>>>> I'd like to include a couple sentences about the different hats >>> at Apache and that votes should be for the benefit of the project and not >>> our employers. >>> >>>>>>>> I'd like to propose that we include text to formally include >>> censor and potential removal for disclosing sensitive information from the >>> private list. >>> >>>>>>>> I'd like to propose branch committers. It has helped Hadoop a >>> lot to enable people to work on development branches for large features >>> before they are given general committership. It is better to have the >>> branch work done at Apache and be visible than having large branches come >>> in late in the project. >>> >>>>>>>> Requirements for each topic (each could be consensus, lazy >>> consensus, lazy majority, lazy 2/3's) >>> >>>>>>>> >>> >>>>>>>> Add committer >>> >>>>>>>> Remove committer >>> >>>>>>>> Add PMC >>> >>>>>>>> Remove PMC >>> >>>>>>>> Accept design proposal >>> >>>>>>>> Add subproject >>> >>>>>>>> Remove subproject >>> >>>>>>>> Release (can't be lazy consensus) >>> >>>>>>>> Modifying bylaws >>> >>>>>>>> >>> >>>>>>>> Thoughts? Missing questions? >>> >>>>>>>> >>> >>>>>>>> .. Owen >>> >>>> >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> Ryan Blue >>> >>>> Databricks >>> >>
