Hi: I think this is a very good idea. In the native pulsar warehouse, there are some interfaces that have no default implementation classes, such as ConsumerInterceptor, ProducerInterceptor. Users must implement the interface themselves, which increases the cost of use. If there is a plug-in library mentioned in this PIP, we can provide some common interface implementations in the plug-in library, such as mertics collection interceptor for producer and consumer.
Best Regards, AuroraTwinkle xiangying meng <xiangyingme...@gmail.com> 于2024年7月23日周二 11:25写道： > > Hello Wen Zhi, > > This project is a new repository similar to a connector, and it will > not make any modifications to Pulsar itself. [0] > > Its code part is simply a plugin library, based on Pulsar's external > interfaces, providing a rich set of implementations. It only retains a > "curated list", which is a series of pointers to personalized features > and experimental features in personal repositories for reference and > inspiration. If these experimental features are recognized by other > users, they can also be gradually merged into the Pulsar main > repository. However, it will not fork the content of the Pulsar > repository for modification in the new repository. > This is the improvement we made after discussion in the last email. I > will synchronize it to the proposal later. > Finally, thank you for your valuable suggestions, and we look forward > to your continued participation. > > [0] - https://github.com/apache/pulsar-connectors > > Best Regards, > Xiangying > > On Tue, Jul 23, 2024 at 10:36 AM WenZhi Feng <thetumb...@apache.org> wrote: > > > > Hi, xiangying, > > First of all, thanks for driving the proposal. > > I have some questions. > > 1. Will this repository maintained by committers/pmc? If the answer is > > yes, there will be too many new modules introduced into pulsar and hard to > > maintain. If the answer is no, i think we have better not to include it > > into pulsar official repository. > > 2. For those plugin-in feature, we may create a new repository like > > various connectors for pulsar. But for those core code change on pulsar, > > how can we seperate it with pulsar repository? > > It is thrilling that users contribute various new features to apache > > community, enriching the ecosystem. I think we should find out a good way > > to realize it. > > > > Thanks, > > Wenzhi Feng. > > > > On 2024/07/22 17:40:24 xiangying meng wrote: > > > Thank you for your feedback, let's further clarify this matter. > > > > > > The original design of this proposal is a new contributor repository, > > > and provides two ways to contribute. > > > The first one: > > > Contributors contribute through PR and accept code review, and can > > > only be merged into the main branch after reviewed. The code merged in > > > this way is various plug-ins based on Pulsar's external interface. It > > > does not contain any pulsar code, only Pulsar's dependencies. > > > The second one: > > > Features that require modifications to the Pulsar main repository code > > > and may conflict with later versions. My initial idea was for > > > contributors to create branches and then write the commit id and > > > features description into the document after the PR is reviewed. > > > > > > However, after the discussion just now, I agree that the second one > > > may not be maintained in the branch of the new repository, but in the > > > contributor's personal repository. > > > > > > Based on the above content, let's continue to look at the question you > > > raised. > > > > > > >- How can the Pulsar community maintain a fork of Pulsar itself ? > > > > > > If it is just plug-in library and a feature list document of the > > > features in personal repository, then there is no need to fork Pulsar > > > itself. > > > > > > >- Do you want to "cut releases" out of this fork ? Who is going to > > > >validate them ? Who is responsible for security bugs? > > > > > > We will only need to maintain a documents of feature and plugins list > > > that have been reviewed by PR. We will not make changes to the Pulsar > > > code itself. There is no fork of Pulsar. It will have its own version > > > release process, just like Pulsar. > > > > > > -> Who is going to use this code ? > > > > > > IMO, providing various implementation plugins based on Pulsar's > > > external interface is meaningful in itself. While introducing Pulsar's > > > own dependencies, introducing plugin libraries to use various already > > > implemented plugins can reduce development costs. I think some people > > > will be happy to use them. > > > > > > As for the other functions in the document, maintained in personal > > > repositories, they can be used as a reference and provided to > > > companies that are capable of solving security issues and bugs. > > > > > > Best regards, > > > Xiangying > > > > > > On Tue, Jul 23, 2024 at 12:03 AM Enrico Olivelli <eolive...@gmail.com> > > > wrote: > > > > > > > > Il giorno lun 22 lug 2024 alle ore 17:34 xiangying meng < > > > > xiangyingme...@gmail.com> ha scritto: > > > > > > > > > >thanks for your proposal, we need more and more energy in the > > > > > >project. > > > > > > > > > > Yes, it requires a lot of participation, but sometimes we also need a > > > > > start. > > > > > > > > > > >Also we allow only "committers" to commit patches to the repositories > > > > > that are under the responsibility of the Apache Pulsar PMC, I don't > > > > > know > > > > > how we can make this work with a "official fork" > > > > > > > > > > >An alternative is to have a "curated list" of links to personal > > > > > >projects, > > > > > but we keep the responsibility over the code on the code owners and > > > > > not on the Pulsar PMC > > > > > > > > > > Yes, this is a "curated list" link, but it points to the branch and > > > > > commit ID created by the contributor himself. In addition, for some > > > > > interface implementation classes, they can be placed in the main > > > > > branch of the new contributor's repository and then added to the > > > > > "curated list". Only committors or maintainers can add content to the > > > > > "curated list". I have detailed these in the proposal. > > > > > > > > > > But no matter which method, each function will be marked in the > > > > > document, that is, in the "Featured List" who contributed it. I > > > > > understand that this feature is more of a reference feature. Allow > > > > > users to choose these customized and risky features to customize their > > > > > own versions. This is very useful for some large companies with > > > > > development capabilities. At the same time, its responsibility should > > > > > belong to the user, although we will also point out who is the > > > > > contributor of each feature he chooses. But when using innovative or > > > > > customized features, users should also be clear about the risks. I > > > > > also wrote about the risks of contributor repositories in the > > > > > proposal. > > > > > > > > > > > > > The main point is that we cannot grant "write" permissions to > > > > repositories > > > > owned by the Pulsar project (the PMC) > > > > to people who are not "committers". This is a legal thing, no > > > > exceptions. > > > > > > > > So any contribution must be in the form of a PR, like for any other > > > > patches > > > > contributes to one of the github repositories owned by the Pulsar > > > > project > > > > > > > > A couple of questions: > > > > - How can the Pulsar community maintain a fork of Pulsar itself ? > > > > - Do you want to "cut releases" out of this fork ? Who is going to > > > > validate > > > > them ? Who is responsible for security bugs ? > > > > - Who is going to use this code ? > > > > > > > > I think that when you want to contribute a feature, you can do it with > > > > a PR > > > > or with the PIP process (if it is bigger), and it is fine to mark it as > > > > "experimental". > > > > > > > > Then the community decides whether to support it or not. > > > > > > > > When a patch is committed to our repository then it is the > > > > responsibility > > > > of all of us (the PMC and the committers) to maintain it almost forever, > > > > unless we decide to delete it. > > > > > > > > This is a legal thing, we cannot accumulate code without being > > > > responsible > > > > for that. > > > > > > > > I am happy to clarify more if needed, and I hope that other in the > > > > community can help in understanding if this is doable of not > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > In addition, thank you for your valuable feedback. > > > > > > > > > > Best regards > > > > > Xiangying > > > > > > > > > > On Mon, Jul 22, 2024 at 10:55 PM Enrico Olivelli <eolive...@gmail.com> > > > > > wrote: > > > > > > > > > > > > Hello, > > > > > > thanks for your proposal, we need more and more energy in the > > > > > > project. > > > > > > > > > > > > I don't think that this is a good idea. > > > > > > > > > > > > The main reason is that as an ASF project we must guarantee the > > > > > > quality > > > > > and > > > > > > especially the security of what we provide to the public. > > > > > > > > > > > > Such kinds of repositories tend to become full of stale code that > > > > > > can > > > > > > become a source of security issues. > > > > > > > > > > > > Also we allow only "committers" to commit patches to the > > > > > > repositories > > > > > that > > > > > > are under the responsibility of the Apache Pulsar PMC, I don't know > > > > > > how > > > > > we > > > > > > can make this work with a "official fork" > > > > > > > > > > > > Maybe I misunderstood the goal ? > > > > > > > > > > > > An alternative is to have a "curated list" of links to personal > > > > > > projects, > > > > > > but we keep the responsibility over the code on the code owners and > > > > > > not > > > > > on > > > > > > the Pulsar PMC > > > > > > > > > > > > > > > > > > Best regards > > > > > > Enrico Olivelli > > > > > > > > > > > > > > > > > > > > > > > > Il giorno lun 22 lug 2024 alle ore 15:15 steven lu < > > > > > lushiji2...@gmail.com> > > > > > > ha scritto: > > > > > > > > > > > > > We think this is a very good solution, > > > > > > > Our XHS team(@liangyepianzhou <https://github.com/liangyepianzhou> > > > > > > > @AuroraTwinkle <https://github.com/AuroraTwinkle> @StevenLuMT > > > > > > > <https://github.com/StevenLuMT> @cai152 > > > > > > > <https://github.com/cai152>) > > > > > will > > > > > > > submit a few to pulsar-java-contrib > > > > > > > <https://github.com/StevenLuMT/pulsar-java-contrib>( > > > > > > > https://github.com/StevenLuMT/pulsar-java-contrib) as a primer to > > > > > > > let > > > > > > > everyone know the role of this library > > > > > > > > > > > > > > xiangying meng <xiangyingme...@gmail.com> 于2024年7月22日周一 16:33写道： > > > > > > > > > > > > > > > Hello Pulsar Community, > > > > > > > > > > > > > > > > I hope this message finds you well. I'm reaching out to propose > > > > > > > > the > > > > > > > > establishment of a Pulsar Contributor Repository. > > > > > > > > > > > > > > > > This new initiative is designed to provide a dedicated space for > > > > > > > > experimental and community-driven features that complement our > > > > > > > > core > > > > > > > > offerings. This repository would serve as a space for non-core > > > > > > > > features and customizations, allowing for greater flexibility > > > > > > > > and > > > > > > > > community involvement. > > > > > > > > > > > > > > > > The proposal can be viewed at this GitHub link: > > > > > > > > https://github.com/apache/pulsar/pull/23061/files > > > > > > > > > > > > > > > > I'm looking forward to your thoughts and feedback. Please feel > > > > > > > > free > > > > > to > > > > > > > > share them on the Pulsar Developer Mailing List. > > > > > > > > > > > > > > > > Thank you in advance for your time and consideration. > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > Xiangying > > > > > > > > > > > > > > > > > > > > > > >
