I found I just misunderstood the "checklist" you mean. I thought it's more like a "summary" of a proposal. So I thought you wanted the reviewers to give a summary list and select which of them are understood. But why do we need a checklist? Is there any reason that any item of the list is not selected?
Thanks, Yunze On Wed, May 10, 2023 at 3:32 PM Asaf Mesika <asaf.mes...@gmail.com> wrote: > > Hi Yunze, > > Thanks for the feedback. > > I re-read your comments 3 times and I can't seem to be able to understand > your key points in the matter of the checklist, so I have some > clarification questions: > > 1. You said you reviewed PIP-261, remembered the checklist proposal, but > couldn't add it. Can you explain why? > 2. Why would the author of a PIP give you a checklist for their vote? Can > you please expand on that? > I completely agree if the author of PIP needs to add a checklist it > will burden, hence I don't see the reason for it and didn't suggest it. > 3. You say you want the process of PIP to be more friendly to contributors. > a) Can you please explain which changes you propose to make it more > friendly? > b) The checklist is for the voters (mainly PMC members), not the PIP > authors. Why would adding the checklist create any burden for the PIP > author and make the PIP process unfriendly? > > 4. In the 2nd paragraph, if I try to summarize, you say it's hard to avoid > changes between the implementation of the PIP and the PIP it self. Also, > it's hard to review PIP implementation since it's divided to many PRs. > Can you please explain the connection between this and a checklist for > voters on PIP? > > 5. You said a checklist won't solve the key difficulties you described for > a huge PIP. > You are correct. It won't. It's the goal of the checklist to solve > those, at all. > > My main goal in the checklist is to make sure that a person, having > basic Pulsar user knowledge, can read the PIP and fully understand it. > > You think the checklist doesn't serve that goal? > > I think for huge PIPs it's even more important that the PIP will be > coherent for the reader and supply all background knowledge. > > 6. I agree with you that implementation can avoid following the design, but > it's a completely different problem we need to solve, unrelated to the > checklist goal. Let's open a separate discussion for it to brainstorm. > > 7. "A complicated proposal could not be understood by many reviewers. If > the author left the community, it could be a hard job to maintain it." > > This is exactly what I want to avoid. > When you vote +1, you must make sure most people reading it can > understand it. If it's not, let's help the author making it so. It must be > the minimum bar for any PIP. > The checklist is to remind you of that. > If the design can be easily understandable, you just made the > implementation x10 easier to follow and maintain when the authors leave the > project. > > > > > > On Tue, May 9, 2023 at 9:39 PM Yunze Xu <y...@streamnative.io.invalid> > wrote: > > > I cannot agree more with Dave's comments. > > > > I just reviewed PIP-261 and PIP-264 yesterday. When I gave +1 to > > PIP-261, I recalled this thread so I'm wondering if I can add a > > checklist. Eventually, I did not do that. IMO, it's the author's > > responsibility to give a checklist for authors to choose for his/her > > proposal. However, it burdens the new contributors to the community. > > PIPs should be more friendly to new contributors. That's also my > > perspective to Rajan's concern: we should still require a PIP for > > changes of metrics or configurations, but the process should be more > > friendly to new contributors. > > > > When I reviewed PIP-264, I recalled PIP-45 and PIP-192 as well, while > > PIP-264 is much more huge than them. Accidentally, I was developing > > KoP for 2.8.0 (not released) when PIP-45 was in progress. It's really > > annoying to see the interfaces changed again and again in the master > > branch. The partner developers maintain their own version of Pulsar > > based on 2.6.x. It's also annoying for them to cherry-pick PRs from > > the master branch. PIP-192 is also a huge proposal. There are so many > > PRs for a proposal. From what I know, It seems the design was slightly > > changed when implementing it. > > > > Adding a checklist cannot solve the key issues for a huge proposal: > > - The design when being voted could be different from PRs > > - The changes could not be easily realized and eventually it was ignored > > - A complicated proposal could not be understood by many reviewers. If > > the author left the community, it could be a hard job to maintain it. > > > > > Being overly dependent on rules is not a replacement for open discussion. > > > > +1. I also hear voices to make some rules for cherry-picking PRs > > during the release process. But it's still necessary to start a > > discussion even if we have any rule. > > > > Thanks, > > Yunze > > > > On Mon, May 8, 2023 at 1:58 AM Dave Fisher <wave4d...@comcast.net> wrote: > > > > > > You asked. Here it is. > > > > > > 1. You brushed aside Enrico’s concerns with that comment. It was not > > subtle. > > > > > > 2. I think the project should pay more attention to Rajan’s concerns > > about new contributors being either ignored or told they need a PIP for > > what seems to them a trivial change. We lose contributors. We need to > > handle that more gently by helping them figure how to better make their PR. > > > > > > 3. For minor PIPs this is too much. Minor PIPs should be easy. > > > > > > 4. For master PIPs like your OTel nothing here is enough. Experience > > with PIP-45 and PIP-192 is that there will be breakage, divergence, and not > > everyone will agree on the result. You worked for 11 months in apparent > > secrecy, yet seemingly ignored Lari’s similar open discussion about scaling > > which occurred in the same time frame. > > > > > > Being overly dependent on rules is not a replacement for open discussion. > > > > > > Sorry if this seems harsh, but this is what I think as an individual. > > > > > > The ASF has a saying “Community over Code” > > > > > > Best, > > > Dave > > > > > > Sent from my iPhone > > > > > > > On May 7, 2023, at 9:55 AM, Asaf Mesika <asaf.mes...@gmail.com> wrote: > > > > > > > > I understand that Dave, and hence I only started a discussion. > > > > What do you think of last reply I made there? > > > > > > > > > > > >> On Sun, May 7, 2023 at 5:31 PM Dave Fisher <wave4d...@comcast.net> > > wrote: > > > >> > > > >> > > > >> > > > >> Sent from my iPhone > > > >> > > > >>>> On Apr 18, 2023, at 5:14 AM, Asaf Mesika <asaf.mes...@gmail.com> > > wrote: > > > >>> > > > >>> The problem I'm trying to solve is: lack of ability to understand > > PIPs. > > > >>> PIPs I had the chance of reading lack: > > > >>> * Background information: It should contain all background > > information > > > >>> necessary to understand the problem and the solution > > > >>> * Clarity: It should be written in a coherent and easy to understand > > way. > > > >>> > > > >>> I thought this could improve using 2 ways: > > > >>> 1. Define a clear template for PIPs - this should solve all the > > missing > > > >>> information. This is in progress. > > > >>> 2. Provide a checklist to verify the +1 voter check those 3 things: > > > >>> background information, clarity, solid technical solution. > > > >>> > > > >>> Both Enrico and Yunze say, if I understand correctly, that the +1 > > voter > > > >>> checks those 3 things implicitly. > > > >>> Yet when I try to learn Pulsar by reading historical PIPs, I find > > some > > > >>> lacking on those things (clarity, background information) making it > > super > > > >>> hard for me to get onboard into Pulsar. > > > >>> > > > >>> Another aspect worth noting is: community increase. In my own > > opinion, > > > >>> documents with clarity and enough background information produce a > > > >> feeling > > > >>> of quality - high quality. Making Pulsar PIPs clear and have all > > > >>> information to understand them will help grow Pulsar adoption. > > > >>> > > > >>> Maybe incremental improvements are better.. If I understand > > correctly, > > > >> both > > > >>> Enrico and Yunze - you are ok with having a summary template, but > > have it > > > >>> non-required? > > > >>> > > > >>> Enrico - Regarding previous suggestions. Root cause - help make > > Pulsar > > > >>> better from my own perspective. Some suggestions may be super bad > > > >>> suggestions and hopefully some will be good :) > > > >>> This specific one - I validated with the PMC members in the weekly > > zoom > > > >>> meeting roughly 3 weeks ago, and got +1 across the board (we had 5 > > > >> people). > > > >>> I did it since I felt it was a touchy subject. > > > >> > > > >> Nothing discussed in that meeting was a decision. PMC Members in the > > > >> community meeting are not making PMC decisions. Decisions are ONLY > > made > > > >> here. Whatever you may think I said my intent was for you to start > > this > > > >> discussion and only that. > > > >> > > > >> Best, > > > >> Dave > > > >> > > > >>> > > > >>> Thanks, > > > >>> > > > >>> Asaf > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>>> On Tue, Apr 18, 2023 at 9:15 AM Yunze Xu > > <y...@streamnative.io.invalid> > > > >>>> wrote: > > > >>>> > > > >>>> Basically I think describing how much work the reviewer did to give > > > >>>> his +1 is good. Just like the vote for a release, each +1 follows > > with > > > >>>> the verifications he did, e.g. here [1] is a vote for Pulsar 2.11.1 > > > >>>> candidate 1: > > > >>>> > > > >>>>> • Built from the source package (maven 3.8.6 OpenJDK 17.0) > > > >>>>> • Ran binary package standalone with pub/sub > > > >>>>> ... > > > >>>> > > > >>>> But I don't think forcing the rule is good. The proposal could > > > >>>> sometimes be not so complicated. From my personal experience, > > > >>>> sometimes I vote my +1 just because I think it's good and there is > > no > > > >>>> serious problem. If you want me to vote again with the checklist, I > > > >>>> might still not have an idea of what I should write, unless there > > is a > > > >>>> template and I filled the template. Only if the proposal is somehow > > > >>>> complicated will the checklist be meaningful, like the PIP-192, > > which > > > >>>> is a very complicated proposal. > > > >>>> > > > >>>>> Moreover, this checklist can ensure that all participants have > > > >>>> thoroughly reviewed the PIP, > > > >>>> > > > >>>> Regarding this point from Xiangying, I want to repeat a similar > > > >>>> thought [2] for the previous discussion. > > > >>>> > > > >>>> IF ANYONE WANT, HE CAN STILL COPY A CHECKLIST FROM OTHERS AND JUST > > > >>>> PERFORM SOME SLIGHTLY CHANGES. > > > >>>> > > > >>>> Forcing a checklist won't change anything if there is a PMC that > > gave > > > >>>> his vote without any careful review. It just makes the rule more > > > >>>> complicated. If you don't trust a PMC, no rule could restrict him. > > > >>>> Rules only make him a better game player. > > > >>>> > > > >>>> In addition, when a reviewer approves a PR, should he add a > > checklist > > > >>>> as well, instead of a simple LGTM or +1? Huge PRs appear more often > > > >>>> than complicated proposals. > > > >>>> > > > >>>> In conclusion, I am +0 to this suggestion. If this suggestion is > > > >>>> passed, I will follow it well. But if I cannot think of a checklist > > > >>>> with a proposal, I will try to be a good vote game player. > > > >>>> > > > >>>> [1] > > https://lists.apache.org/thread/13xmt4jdwmlo1mo5dhkxlg9pnkfdwjjj > > > >>>> [2] > > https://lists.apache.org/thread/o0vw1dfoo84pscfd46gdm3sm9mvovmr2 > > > >>>> > > > >>>> Thanks, > > > >>>> Yunze > > > >>>> > > > >>>>> On Mon, Apr 17, 2023 at 3:48 PM PengHui Li <peng...@apache.org> > > wrote: > > > >>>>> > > > >>>>> I don't think it will bring more burden on reviewers. > > > >>>>> It will only provide a checklist for reviewers before > > > >>>>> you vote +1 or -1. It could be done in 1 minute if you > > > >>>>> did a great proposal review. Of course, if you are > > > >>>>> missing some aspects that should be reviewed, > > > >>>>> This will make the reviewer spend more time reviewing > > > >>>>> the missing items, but it is valuable. > > > >>>>> > > > >>>>> I don't think this proposal is accusing PMCs, but PMCs > > > >>>>> might also miss some items. The checklist can help PMCs > > > >>>>> to avoid missing items. Actually, I think every PMC has > > > >>>>> checklist for a proposal review. It might be recorded in > > > >>>>> a tiny notebook, or in his brain. Now, the proposal provides > > > >>>>> a way to share your experience of proposal review. > > > >>>>> > > > >>>>> And we are actually doing the same thing in the voting of > > > >>>>> release. Everyone will provide a list of what they have > > > >>>>> verified with +1 or -1. > > > >>>>> > > > >>>>> Regards, > > > >>>>> Penghui > > > >>>>> > > > >>>>> > > > >>>>> On Mon, Apr 17, 2023 at 10:37 AM Xiangying Meng < > > xiangy...@apache.org> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Hi, Asaf > > > >>>>>> This is a great suggestion. I believe one significant advantage is > > > >> that > > > >>>>>> it can help newcomers better understand the voting process and how > > > >>>>>> decisions are made. > > > >>>>>> The checklist can serve as a reference framework, > > > >>>>>> assisting new members in becoming familiar with the project's > > voting > > > >>>>>> requirements and standards more quickly, > > > >>>>>> thereby improving the overall participation and transparency of > > the > > > >>>>>> project. > > > >>>>>> > > > >>>>>> Moreover, this checklist can ensure that all participants have > > > >>>> thoroughly > > > >>>>>> reviewed the PIP, > > > >>>>>> resulting in higher-quality PIPs. > > > >>>>>> Although introducing a checklist may bring some additional burden, > > > >>>>>> in the long run, it contributes to the project's robust > > development > > > >> and > > > >>>>>> continuous improvement. > > > >>>>>> > > > >>>>>> Thanks > > > >>>>>> Xiangying > > > >>>>>> > > > >>>>>> > > > >>>>>> On Sun, Apr 16, 2023 at 11:23 PM Enrico Olivelli < > > eolive...@gmail.com > > > >>> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> Asaf, > > > >>>>>>> I understand your intent. > > > >>>>>>> > > > >>>>>>> I think that when anyone casts a +1, especially with '(binding)' > > they > > > >>>>>> know > > > >>>>>>> well what they are doing. > > > >>>>>>> It is not an 'I like it', but it is an important assumption of > > > >>>>>>> responsibility. > > > >>>>>>> This applies to all the VOTEs. > > > >>>>>>> > > > >>>>>>> Requiring this checklist may be good in order to help new comers > > to > > > >>>>>>> understand better how we take our decisions. > > > >>>>>>> > > > >>>>>>> If you feel that currently there are people who cast binding > > votes > > > >>>>>> without > > > >>>>>>> knowing what they do...then I believe that it is kind of a > > serious > > > >>>> issue. > > > >>>>>>> > > > >>>>>>> It happened a few times recently that I see this sort of ML > > threads > > > >>>>>> about > > > >>>>>>> 'the PMC is not doing well', 'we want to retire people in the > > > >>>> PMC...', > > > >>>>>> 'PMC > > > >>>>>>> members vote on stuff without knowing what they do'... > > > >>>>>>> > > > >>>>>>> I wonder what is the root cause of this. > > > >>>>>>> > > > >>>>>>> Back to he original question, my position it: > > > >>>>>>> +1 to writing a clear and very brief summary of the consideration > > > >>>> you hBe > > > >>>>>>> to take before casting your vote. > > > >>>>>>> -1 to requiring this checklist when we cast a vote > > > >>>>>>> > > > >>>>>>> Thanks > > > >>>>>>> Enrico > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> Il Dom 16 Apr 2023, 15:47 Asaf Mesika <asaf.mes...@gmail.com> ha > > > >>>>>> scritto: > > > >>>>>>> > > > >>>>>>>> Would love additional feedback on this suggestion. > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Fri, Mar 31, 2023 at 4:19 AM PengHui Li <peng...@apache.org> > > > >>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> It looks like we can try to add a new section to > > > >>>>>>>>> > > > >>>> https://github.com/apache/pulsar/blob/master/wiki/proposals/PIP.md > > > >>>>>>>>> like "Review the proposal" and it is not only for PMCs, all the > > > >>>>>>> reviewers > > > >>>>>>>>> can follow the checklist > > > >>>>>>>>> to cast a solemn vote. > > > >>>>>>>>> > > > >>>>>>>>> And I totally support the motivation of this discussion. > > > >>>>>>>>> > > > >>>>>>>>> Regards, > > > >>>>>>>>> Penghui > > > >>>>>>>>> > > > >>>>>>>>> On Fri, Mar 31, 2023 at 4:46 AM Asaf Mesika < > > > >>>> asaf.mes...@gmail.com> > > > >>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> Hi, > > > >>>>>>>>>> > > > >>>>>>>>>> When you read last year's PIPs, many lack background > > > >>>> information, > > > >>>>>>> hard > > > >>>>>>>> to > > > >>>>>>>>>> read and understand even if you know pulsar in and out. > > > >>>>>>>>>> > > > >>>>>>>>>> First step to fix was to change the PIP is structured: > > > >>>>>>>>>> https://github.com/apache/pulsar/pull/19832 > > > >>>>>>>>>> > > > >>>>>>>>>> In my opinion, when someone votes "+1" and it's binding, they > > > >>>>>>> basically > > > >>>>>>>>>> take the responsibility to say: > > > >>>>>>>>>> > > > >>>>>>>>>> * I read the PIP fully. > > > >>>>>>>>>> * A person having basic Pulsar user knowledge, can read the > > > >>>> PIP and > > > >>>>>>>> fully > > > >>>>>>>>>> understand it > > > >>>>>>>>>> Why? Since it contains all background information necessary > > > >>>> to > > > >>>>>>>>>> understand the problem and the solution > > > >>>>>>>>>> It is written in a coherent and easy to understand way. > > > >>>>>>>>>> * I validated the solution technically and can vouch for it. > > > >>>>>>>>>> Examples: > > > >>>>>>>>>> The PIP adds schema compatibility rules for Protobuf > > > >>>> Native. > > > >>>>>>>>>> I learned / know protobuf well. > > > >>>>>>>>>> I validated the rules written containing all rules > > > >>>>>>> needed > > > >>>>>>>>> and > > > >>>>>>>>>> not containing wrong rules, or missing rules. > > > >>>>>>>>>> > > > >>>>>>>>>> The PIP adds new OpenID Connect authentication. > > > >>>>>>>>>> I learned / know Authentication in Pulsar. > > > >>>>>>>>>> I learned / know OpenID connect > > > >>>>>>>>>> I validated the solution is architecturally > > > >>>> correct > > > >>>>>>> and > > > >>>>>>>>>> sound. > > > >>>>>>>>>> > > > >>>>>>>>>> Basically the PMC member voting +1 on it, basically acts as > > > >>>> Tech > > > >>>>>> Lead > > > >>>>>>>> of > > > >>>>>>>>>> Pulsar for this PIP. > > > >>>>>>>>>> It's a very big responsibility. > > > >>>>>>>>>> It's the only way to ensure Pulsar architecture won't go > > > >>>> haywire > > > >>>>>> over > > > >>>>>>>> the > > > >>>>>>>>>> next few years. > > > >>>>>>>>>> > > > >>>>>>>>>> Yes, it will slow the process down. > > > >>>>>>>>>> Yes, it will be harder to find people to review it like that. > > > >>>>>>>>>> > > > >>>>>>>>>> But, it will raise the bar for PIPs and for Pulsar > > architecture > > > >>>>>>>> overall. > > > >>>>>>>>>> IMO we need that, and it's customary. > > > >>>>>>>>>> > > > >>>>>>>>>> *My suggestion* > > > >>>>>>>>>> When PMC member replies to vote, it will look like this: > > > >>>>>>>>>> > > > >>>>>>>>>> " > > > >>>>>>>>>> +1 (binding) > > > >>>>>>>>>> > > > >>>>>>>>>> [v] PIP has all sections detailed in the PIP template > > > >>>> (Background, > > > >>>>>>>>>> motivation, etc.) > > > >>>>>>>>>> [v] A person having basic Pulsar user knowledge, can read the > > > >>>> PIP > > > >>>>>> and > > > >>>>>>>>> fully > > > >>>>>>>>>> understand it > > > >>>>>>>>>> [v] I read PIP and validated it technically > > > >>>>>>>>>> " > > > >>>>>>>>>> > > > >>>>>>>>>> or > > > >>>>>>>>>> " > > > >>>>>>>>>> -1 (binding) > > > >>>>>>>>>> > > > >>>>>>>>>> I think this PIP needs: > > > >>>>>>>>>> ... > > > >>>>>>>>>> " > > > >>>>>>>>>> > > > >>>>>>>>>> Thanks, > > > >>>>>>>>>> > > > >>>>>>>>>> Asaf > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >> > > > >> > > > > >
