Here are my two cents. It is great to have more protocol handlers. I'd encourage people to maintain the protocol handler themselves rather than pushing to the upstream.
1. It is very hard for the community to maintain all protocol handlers upstream and keep them updated. Similar situations happen to connectors. It is also commonly seen in other projects like Spark and Flink. The creator of the protocol handler is the best person to maintain the protocol handler. You can grow the community for a given protocol handler as well. It is how we build and maintain KoP, AoP, and MoP. 2. gRPC isn't a zero-copy protocol. It is not well suited for implementing high-throughput streaming services. 3. Unlike KoP, MoP and AoP which are implementing well-known messaging protocols, the gRPC protocol handler is not an extension to support a different protocol. It is essentially re-implementing Pulsar's protocol in gRPC. This leads to a serious problem as it is competing with the original protocol. It will confuse the community on what to use. Instead of introducing another protocol handler to re-implement Pulsar's protocol, I would suggest discussing whether gRPC is suitable for Pulsar or how we can involve Pulsar's current protocol. 4. If you want to implement a gRPC based service, an alternative would be adding a Google Pub/Sub protocol handler to support the protocol of Google Pub/Sub. - Sijie On Tue, Jul 27, 2021 at 6:56 AM Christophe Bornet <bornet.ch...@gmail.com> wrote: > > Hi Apache Pulsar community, > > I’m a huge fan of the potential that the pluggable protocol handlers > introduced in Pulsar 2.5 unlock for increasing Pulsar popularity by > reducing the friction to adopt it. This includes both general-purpose APIs > like REST and gRPC, and compatibility layers for other streaming products > like Kafka and AMQP. > > I’ve created a PIP > <https://github.com/apache/pulsar/wiki/PIP-59%3A-gPRC-Protocol-Handler> and > an implementation of a gRPC protocol handler, but review of the pull > request stalled out <https://github.com/apache/pulsar/pull/6512> so I moved > my implementation to my own repository > <https://github.com/cbornet/pulsar-grpc> while I finished it. > > With development and testing effectively complete, I’d like to reopen the > discussion: > > > 1. > > How can I move forward with contributing my gRPC protocol handler to > Apache Pulsar? Is this something that is interesting and useful to the > broader community? > 2. > > More broadly, what policy do we want to establish for Pulsar protocol > handlers and other compatibility layers? As indicated above, I think that > we should provide Pulsar with “batteries included” to make it as easy as > possible for new users to get started. > > > Best regards. > > Christophe
