I like the idea of keeping the suggestions as files in the repo since as you mentioned, it makes the review process using PRs much more streamlined.
I think keeping the status in an MD file and only there (having a single source of truth) will make it less error-prone (people might forget to move between directories) , and also easier to have a single page to view all proposals. The only point worth attending to while drafting the new process is how to avoid neglected out-of-date proposals. If a proposal was declined, who is in charge to update its status in the readme? There is something appealing in not merging a PR proposal, as this status update takes care of itself (borrowing from rust RFC). One bonus item is that I learned through this long discussion thread about Zulip chat :) On Mon, Aug 29, 2022 at 1:55 PM PengHui Li <peng...@apache.org> wrote: > Hi all, > > I will merge https://github.com/apache/pulsar/pull/17270 first and draft a > proposal for > the detailed PIP process changes. And will start a new VOTE thread for the > PIP process change. > > After the proposal(for PIP process change) is approved. I will go back here > to discuss > how to migrate the old PIPs to the codebase(because we need to follow > the new format of PIPs). > > Thanks, > Penghui > > On Fri, Aug 26, 2022 at 11:20 PM PengHui Li <peng...@apache.org> wrote: > > > > Using a directory structure to organize PIP status might make the git > > history a bit less direct because changing state is equivalent with a > > file move instead of a line updated in a file. However, if we do it > > that way, we could have a README.md file organizing PIP metadata and > > linking to the actual PIP file in the directory structure. That > > README.md would also serve as the source of truth for PIP numbers > > because each PIP pointer would get its associated line number. Then, > > concurrent PIPs would need to resolve merge conflicts just before > > merging and only then would they acquire their number. > > > > Oh, get your point, Michael. I think this solution looks better. We can > > also > > add something in README.md and users can also get the complete > > proposal list here. In the future, maybe we can show it on the website. > > > > Best, > > Penghui > > > > On Fri, Aug 26, 2022 at 12:16 PM Michael Marshall <mmarsh...@apache.org> > > wrote: > > > >> > We should move to the codebase 100% the same as the original > >> documentation. > >> > And use another PR to make improvements for it. If it is needed, we > >> should > >> > start with an email. > >> > We need to have historical records. > >> > >> +1 I think this is a great idea. It makes sense to copy them verbatim > >> and then worry about updating them in a second step. > >> > >> Using a directory structure to organize PIP status might make the git > >> history a bit less direct because changing state is equivalent with a > >> file move instead of a line updated in a file. However, if we do it > >> that way, we could have a README.md file organizing PIP metadata and > >> linking to the actual PIP file in the directory structure. That > >> README.md would also serve as the source of truth for PIP numbers > >> because each PIP pointer would get its associated line number. Then, > >> concurrent PIPs would need to resolve merge conflicts just before > >> merging and only then would they acquire their number. > >> > >> - Michael > >> > >> On Thu, Aug 25, 2022 at 9:15 PM PengHui Li <peng...@apache.org> wrote: > >> > > >> > Hi Xiangying, > >> > > >> > > We can classify the pips under these folders according to the pulsar > >> > modules, instead of just placing these pips under these folders in an > >> > incrementing sequence number. > >> > > >> > I think it's not easy to do this. A proposal can have changes related > to > >> > multiple > >> > modules(Broker, Metadata, Client). > >> > > >> > Thanks, > >> > Penghui > >> > > >> > On Fri, Aug 26, 2022 at 9:20 AM Xiangying Meng <xiangy...@apache.org> > >> wrote: > >> > > >> > > Hi Penghui > >> > > Thanks for you start this discussion. IMO, It is also a good way for > >> > > beginners to learn the design and implementation of each module of > >> Pulsar. > >> > > > 3. /wiki/pip/accepted - for PIPs that have been accepted and are > >> works in > >> > > progress > >> > > > 4. /wiki/pip/complete - for PIPs that have been completed. > >> > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but then > >> rejected or > >> > > abandoned. > >> > > We can classify the pips under these folders according to the pulsar > >> > > modules, instead of just placing these pips under these folders in > an > >> > > incrementing sequence number. > >> > > > >> > > In this way, readers can create a new local branch dedicated to > >> reading and > >> > > annotating proposals for themselves to read proposals they are > >> interested > >> > > in and write their own understanding and comments anytime and > >> anywhere. > >> > > Thanks, > >> > > Xiangying Meng > >> > > > >> > > On Fri, Aug 26, 2022 at 12:23 AM PengHui Li <peng...@apache.org> > >> wrote: > >> > > > >> > > > Hi Dave, > >> > > > > >> > > > > Let’s outline how PIPs are currently working and then discuss > >> changes. > >> > > > > >> > > > Yes, I will prepare for the changes. > >> > > > This is the documentation for how PIPs are currently working: > >> > > > > >> > > > > >> > > > >> > https://github.com/apache/pulsar/pull/17270/files#diff-a56445d038f8a3df4c74724076c62437915f091437b4e26a1c5aada7184dcffd > >> > > > The mailing list discussion: > >> > > > https://lists.apache.org/thread/m8dr0hz7qn7rkd48bcp430lcq2q3674g > >> > > > > >> > > > Anyway, I will start a new discussion with the new changes to the > >> current > >> > > > process. > >> > > > > >> > > > > I’m not sure what is meant by putting the PIP into the > “codebase”. > >> > > > > Is the proposal to create PIPs here? > >> > > > https://github.com/apache/pulsar/tree/master/wiki > >> > > > > >> > > > Just move out from > >> https://github.com/apache/pulsar/tree/master/wiki to > >> > > > Pulsar codebase /wiki/proposals > >> > > > > >> > > > > I think that a directory structure / convention could be used > for > >> pip > >> > > > status: > >> > > > > >> > > > > 1. /wiki/pip/discussion - for PIPs being discussed and > specified. > >> > > > > 2. /wiki/pip/proposed - for PIPs ready to be formally DISCUSSED > >> and > >> > > VOTED > >> > > > > 3. /wiki/pip/accepted - for PIPs that have been accepted and are > >> works > >> > > in > >> > > > progress > >> > > > > 4. /wiki/pip/complete - for PIPs that have been completed. > >> > > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but then > >> rejected > >> > > or > >> > > > abandoned. > >> > > > > >> > > > I think it's a good point, I don't see any obvious cons. > >> > > > > >> > > > Thanks, > >> > > > Penghui > >> > > > > >> > > > On Thu, Aug 25, 2022 at 11:40 PM Dave Fisher <w...@apache.org> > >> wrote: > >> > > > > >> > > > > > >> > > > > > On Aug 23, 2022, at 10:22 AM, Rajan Dhabalia < > >> dhabalia...@gmail.com> > >> > > > > wrote: > >> > > > > > > >> > > > > > Hi, > >> > > > > > > >> > > > > >>>> I think we can move all the PIPs to the codebase and the > new > >> > > > proposal > >> > > > > > and proposal without any reviews should happen with a PR > first. > >> So > >> > > that > >> > > > > we > >> > > > > > can review and comment easily. > >> > > > > > > >> > > > > > I didn't understand this part. You want one to create a PR > >> before > >> > > > > > submitting a proposal? That's clearly not a good idea because > >> if the > >> > > > PIP > >> > > > > > approach will change then the entire development effort will > be > >> > > wasted > >> > > > > and > >> > > > > > that's the whole purpose of PIP. I guess creating PIP into an > >> issue > >> > > and > >> > > > > > discussing the issue is definitely working and it's an easier > >> way to > >> > > > > > discuss quickly rather than discussing over email threads. > >> > > > > > > >> > > > > > Let's not change this practice without good discussion and > >> agreement > >> > > > from > >> > > > > > the community. > >> > > > > > >> > > > > Agreed let’s have a PIP Discussion here to carefully outline how > >> the > >> > > PIP > >> > > > > process will change. I don’t think that a new PIP should be > overly > >> > > > planned > >> > > > > or implemented before the idea is more fully discussed and > >> accepted. > >> > > The > >> > > > > Apache Way always works best with small incremental and > reversible > >> > > steps. > >> > > > > > >> > > > > Let’s outline how PIPs are currently working and then discuss > >> changes. > >> > > > I’m > >> > > > > not sure what is meant by putting the PIP into the “codebase”. > >> > > > > > >> > > > > Is the proposal to create PIPs here? > >> > > > > https://github.com/apache/pulsar/tree/master/wiki > >> > > > > > >> > > > > I think that a directory structure / convention could be used > for > >> pip > >> > > > > status: > >> > > > > > >> > > > > 1. /wiki/pip/discussion - for PIPs being discussed and > specified. > >> > > > > 2. /wiki/pip/proposed - for PIPs ready to be formally DISCUSSED > >> and > >> > > VOTED > >> > > > > 3. /wiki/pip/accepted - for PIPs that have been accepted and are > >> works > >> > > in > >> > > > > progress > >> > > > > 4. /wiki/pip/complete - for PIPs that have been completed. > >> > > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but then > >> rejected > >> > > or > >> > > > > abandoned. > >> > > > > > >> > > > > Regards, > >> > > > > Dave > >> > > > > > >> > > > > > > >> > > > > > Thanks, > >> > > > > > Rajan > >> > > > > > > >> > > > > > On Thu, Aug 18, 2022 at 8:27 AM PengHui Li < > peng...@apache.org> > >> > > wrote: > >> > > > > > > >> > > > > >> Hi all, > >> > > > > >> > >> > > > > >> Currently, the new proposal will be added to the issue list > >> and then > >> > > > > shared > >> > > > > >> link in the email > >> > > > > >> to request the proposal review. It's really hard to review a > >> long > >> > > > > proposal > >> > > > > >> if you want to comment > >> > > > > >> in detail. > >> > > > > >> > >> > > > > >> Here is an example: > >> > > > > >> > >> > > > https://github.com/apache/pulsar/issues/16763#issuecomment-1219606491 > >> > > > > >> This seems very unintuitive. > >> > > > > >> > >> > > > > >> I think we can move all the PIPs to the codebase and the new > >> > > proposal > >> > > > > and > >> > > > > >> proposal without > >> > > > > >> any reviews should happen with a PR first. So that we can > >> review and > >> > > > > >> comment easily. > >> > > > > >> Certainly, all the votes should happen on the mailing list. > >> And we > >> > > can > >> > > > > also > >> > > > > >> discuss the > >> > > > > >> proposal on the mailing list. > >> > > > > >> > >> > > > > >> Following this way, we don't need to sync the PIPs from the > >> issue to > >> > > > the > >> > > > > >> wiki page. > >> > > > > >> We can just add a link that points to the PIPs dir to the > >> > > contribution > >> > > > > >> guide or README. > >> > > > > >> > >> > > > > >> We have another pain point about the duplicated PIP number. > We > >> can > >> > > > > maintain > >> > > > > >> a file, a list of > >> > > > > >> all the proposal contains the approved, in-review, drafting. > >> Before > >> > > > > >> creating a proposal, we should > >> > > > > >> have a discussion first on the mailing list, just get > feedback > >> on > >> > > the > >> > > > > >> motivation. If there are no objections, > >> > > > > >> the proposal owner can add a line to the file with the PIP > >> number > >> > > > > through a > >> > > > > >> PR, like PIP-123: xxx (Under Discussion). > >> > > > > >> So that we can prevent the duplicated PIP number(which will > >> conflict > >> > > > if > >> > > > > >> someone merged first). > >> > > > > >> After the PR is merged, we can send out a new PR to add the > >> > > proposal. > >> > > > > >> > >> > > > > >> Thanks, > >> > > > > >> Penghui > >> > > > > >> > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >
