Re: Connectors package registry

2021-06-04 Thread Jonathan Ellis
Do we have consensus on metadata repo + generated docs page to start with? On Sat, May 29, 2021 at 8:44 AM Jonathan Ellis wrote: > On Fri, May 28, 2021 at 2:55 PM Sijie Guo wrote: > >> An alternative is to just add a table to the connectors documentation page >> like how we add third-party cli

Re: Connectors package registry

2021-05-29 Thread Jonathan Ellis
On Fri, May 28, 2021 at 2:55 PM Sijie Guo wrote: > An alternative is to just add a table to the connectors documentation page > like how we add third-party clients. > http://pulsar.apache.org/docs/en/client-libraries/#third-party-clients > > It only refers to the Github repos for those connectors

Re: Connectors package registry

2021-05-28 Thread Sijie Guo
On Thu, May 27, 2021 at 7:54 PM Jonathan Ellis wrote: > It sounds like you're envisioning an "Apple Store" model where every > submission is rigorously tested and vetted. That is certainly an option, > but since the PMC gets to define what the rules are, it's also an option to > say, "this index

Re: Connectors package registry

2021-05-27 Thread Jonathan Ellis
It sounds like you're envisioning an "Apple Store" model where every submission is rigorously tested and vetted. That is certainly an option, but since the PMC gets to define what the rules are, it's also an option to say, "this index is provided as a community service with no guarantees of qualit

Re: Connectors package registry

2021-05-27 Thread Sijie Guo
On Thu, May 27, 2021 at 1:17 PM Jonathan Ellis wrote: > On Thu, May 27, 2021 at 2:38 PM Sijie Guo wrote: > > > Agreed that the main problem is about discovering the existing pre-built > > Pulsar connectors. I don't think the PMC should involve hosting and > > managing external connectors because

Re: Connectors package registry

2021-05-27 Thread Jonathan Ellis
On Thu, May 27, 2021 at 2:38 PM Sijie Guo wrote: > Agreed that the main problem is about discovering the existing pre-built > Pulsar connectors. I don't think the PMC should involve hosting and > managing external connectors because it will put the PMC in the situation > in handling licensing iss

Re: Connectors package registry

2021-05-27 Thread Sijie Guo
Agreed that the main problem is about discovering the existing pre-built Pulsar connectors. I don't think the PMC should involve hosting and managing external connectors because it will put the PMC in the situation in handling licensing issues that I think we should avoid. All the ASF accepted con

Re: Connectors package registry

2021-05-26 Thread Jonathan Ellis
On Wed, May 26, 2021 at 10:45 AM Andrey Yegorov wrote: > My notes from the community meeting. Jerry, Matteo, and I talked about this > idea: > Thanks for the proposal and the notes, Andrey! I think a connector registry will be very useful as Pulsar adoption grows. * major problems are: > --

Re: Connectors package registry

2021-05-26 Thread Andrey Yegorov
My notes from the community meeting. Jerry, Matteo, and I talked about this idea: * major problems are: -- the process of adding new stuff (approval, review, quality control/approval stuff) -- what to do with stuff that is not under apache umbrella (as in: code ownership, not license). ASF pos

Re: Connectors package registry

2021-05-25 Thread Andrey Yegorov
I do agree that hosting of the binaries is not an issue. Discovering the binaries is. Proposed repo should not host the binaries but the urls/checksums etc. and serve as a simple data storage for the CLI to use. Having multiple parties publishing their connectors is fine but it makes it hard for t

Re: Connectors package registry

2021-05-25 Thread Jerry Peng
Hello Andrey, Thank you for bringing this up! This is definitely an important issue! All of the connector binaries are already hosted on Maven central thus I don't think hosting the binaries is an issue. Perhaps the key problem here is about discovery. My thoughts: 1. We should document clea

Connectors package registry

2021-05-25 Thread Andrey Yegorov
Hello, As Pulsar becomes increasingly popular, we will have to deal with a larger userbase looking to deploy Pulsar in a wider array of use cases, interfacing with a more diverse set of other components. To help with this, we should create a plan as a project to help community members publish and