For K8s Should we use something like this https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/kafka_broker_filter#config-network-filters-kafka-broker A pulsar filter in envoy would be great!
On Sat, 28 Aug 2021 at 23:10, Enrico Olivelli <eolive...@gmail.com> wrote: > Joe, > Thanks for sharing your knowledge > > > Il Sab 28 Ago 2021, 18:34 Joe F <joefranc...@gmail.com> ha scritto: > > > > > To give some history and context, Pulsar proxy was meant to be a > barebones > > TCP proxy when it was built. It's sole reason to exist was to forward > > network traffic to the right host. Discovery/authn/z was a dependency . > > The way it came around, it was for a narrow use case (and it was not for > > k8s), and a quick and dirty solution > > > > I am all for making the life of protocol developers easier. I'm just > > concerned that something that was hastily done is now evolving into a > > full-fledged service in a piece-meal, spaghetti style. (There was > another > > proposal a few days ago for dynamic proxy roles.) > > > > We can let the Proxy evolve like the proverbial ball of mud, or put some > > thought into it > > > > I am going to post a PIP, a full design document on which we can discuss > the details > > I am sure that now the community will be able to discuss and get to adding > another future proof feature > > > Regards > Enrico > > > > -j > > > > On Fri, Aug 27, 2021 at 2:04 PM Michael Marshall <mikemars...@gmail.com> > > wrote: > > > >> +1 Thanks for your proposal, Enrico. > >> > >> I completely agree that the Pulsar Proxy is an integral component in a > >> Pulsar cluster running on k8s. Further, considering that the proxy > >> interacts with clients as if it were a broker and that we already > support > >> protocol handlers in the broker, I think it is a logical next step to > add > >> support for protocol handlers in the proxy. > >> > >> I look forward to reviewing the PIP. > >> > >> - Michael > >> > >> On Fri, Aug 27, 2021 at 8:21 AM Enrico Olivelli <eolive...@gmail.com> > >> wrote: > >> > >> > Hello, > >> > Currently we have the ability to add Protocol Handlers to the Pulsar > >> > Broker, this is great, because you can add your code that uses > internal > >> > Pulsar APIs and implement your own protocols. > >> > > >> > When you run Pulsar in k8s (and this is happening more and more) you > >> need > >> > to run the Pulsar proxy. > >> > The Pulsar proxy is put in front of a Pulsar Cluster and allows > clients > >> > outside of the cluster to access with a single endpoint (the proxy may > >> be > >> > replicated, but let's not enter too much into the details). > >> > > >> > As we are doing for the Pulsar Broker I would like to add support for > >> > adding ProtocolHandlers to the Pulsar Proxy service. > >> > The API will be the same, apart from the fact that you have access to > >> the > >> > PulsarProxy object instead of the PulsarBroker. > >> > > >> > It would be great to see this feature, if you have ever come to create > >> your > >> > own Pulsar proxy in front of a ProtocolHandler you had to deal with: > >> > - Broker Discovery > >> > - Authentication > >> > - Authorization > >> > > >> > Reimplementing this, using APIs that are not officially exported by > the > >> > internals of Pulsar code, is very error prone and also it is very > >> difficult > >> > to follow Pulsar evolution. > >> > > >> > PHs for the Proxy will ease the Deployment of Pulsar with PHs as you > do > >> not > >> > need to add other Services/Pods to your cluster. > >> > > >> > If this idea sounds good to you I will be happy to write up a PIP and > >> send > >> > the implementation. > >> > > >> > Enrico > >> > > >> > > > -- *Thanks* *Yuvaraj L*
