Resurrecting this thread. Two points to make: First, it's completely fine for Apache projects to add someone as a committer to maintain a specific part of the tree, and I believe Pulsar has already done this for the .NET driver. So if we agree in principle that a protocol handler or other piece of code should be in-tree, maintainership shouldn't be an obstacle.
Second, I think the right approach is to have a mix -- popular protocols that play well with Pulsar's semantics should be maintained and distributed with Pulsar to provide the least friction possible. In particular, DataStax has seen a lot of interest in our Fast JMS implementation https://github.com/datastax/pulsar-jms and we would like to donate it to Apache Pulsar. On Tue, Jul 27, 2021 at 11:31 PM Sijie Guo <guosi...@gmail.com> wrote: > Here are my two cents. > > It is great to have more protocol handlers. I'd encourage people to > maintain the protocol handler themselves rather than pushing to the > upstream. > > 1. It is very hard for the community to maintain all protocol handlers > upstream and keep them updated. Similar situations happen to > connectors. It is also commonly seen in other projects like Spark and > Flink. The creator of the protocol handler is the best person to > maintain the protocol handler. You can grow the community for a given > protocol handler as well. It is how we build and maintain KoP, AoP, > and MoP. > > 2. gRPC isn't a zero-copy protocol. It is not well suited for > implementing high-throughput streaming services. > > 3. Unlike KoP, MoP and AoP which are implementing well-known messaging > protocols, the gRPC protocol handler is not an extension to support a > different protocol. It is essentially re-implementing Pulsar's > protocol in gRPC. This leads to a serious problem as it is competing > with the original protocol. It will confuse the community on what to > use. Instead of introducing another protocol handler to re-implement > Pulsar's protocol, I would suggest discussing whether gRPC is suitable > for Pulsar or how we can involve Pulsar's current protocol. > > 4. If you want to implement a gRPC based service, an alternative would > be adding a Google Pub/Sub protocol handler to support the protocol of > Google Pub/Sub. > > - Sijie > > On Tue, Jul 27, 2021 at 6:56 AM Christophe Bornet > <bornet.ch...@gmail.com> wrote: > > > > Hi Apache Pulsar community, > > > > I’m a huge fan of the potential that the pluggable protocol handlers > > introduced in Pulsar 2.5 unlock for increasing Pulsar popularity by > > reducing the friction to adopt it. This includes both general-purpose > APIs > > like REST and gRPC, and compatibility layers for other streaming products > > like Kafka and AMQP. > > > > I’ve created a PIP > > <https://github.com/apache/pulsar/wiki/PIP-59%3A-gPRC-Protocol-Handler> > and > > an implementation of a gRPC protocol handler, but review of the pull > > request stalled out <https://github.com/apache/pulsar/pull/6512> so I > moved > > my implementation to my own repository > > <https://github.com/cbornet/pulsar-grpc> while I finished it. > > > > With development and testing effectively complete, I’d like to reopen the > > discussion: > > > > > > 1. > > > > How can I move forward with contributing my gRPC protocol handler to > > Apache Pulsar? Is this something that is interesting and useful to the > > broader community? > > 2. > > > > More broadly, what policy do we want to establish for Pulsar protocol > > handlers and other compatibility layers? As indicated above, I think > that > > we should provide Pulsar with “batteries included” to make it as easy > as > > possible for new users to get started. > > > > > > Best regards. > > > > Christophe > -- Jonathan Ellis co-founder, http://www.datastax.com @spyced
