The trouble is that the producer and consumer clients need to discover the broker hostnames and address the individual brokers directly. There is an advertised.listeners setting that will allow you to tell clients to connect to external proxy hostnames instead of your internal ones, but those proxies will need to know how to map directly from an advertised hostname to an internal kafka broker hostname. You’ll need some logic in your proxy to do that routing.
P.S. I’ve not actually set this up before, but this is my understanding :) On Thu, Aug 30, 2018 at 7:16 AM Dan Markhasin <minimi...@gmail.com> wrote: > Usually for such a use case you'd have a physical load balancer box (F5, > etc.) in front of Kafka that would handle the SSL termination, but it > should be possible with NGINX as well: > > > https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-tcp/ > > On Fri, 24 Aug 2018 at 18:35, Jack S <jacksilver1...@gmail.com> wrote: > > > Thanks Ryanne. > > > > That's one of the options we had considered. I was hoping to keep > solution > > simple and efficient. With HTTP proxy, we would have to worry about > > configurations, scalability, and operation. This is probably true with > > proxy solution as well, but at least my thinking was that deploying proxy > > would be more standard with less management effort on our side. Also, we > > are very familiar with Kafka usual producer/consumer usage, operation, > etc. > > and could re-use much of our producer and consumer infrastructure that we > > currently use internally. > > > > Having said that, this is where I was hoping to hear and get feedback > from > > community - what people have deployed with such use case and any > learnings > > and suggestions. > > > > On Fri, Aug 24, 2018 at 7:42 AM Ryanne Dolan <ryannedo...@gmail.com> > > wrote: > > > > > Can you use a Kafka HTTP proxy instead of using the Kafka protocol > > > directly? > > > > > > Ryanne > > > > > > On Thu, Aug 23, 2018, 7:29 PM Jack S <jacksilver1...@gmail.com> wrote: > > > > > > > Hello, > > > > > > > > We have a requirement for opening Kafka on WAN where external > producers > > > and > > > > consumers need to be able to talk to Kafka. I was able to get > Zookeeper > > > and > > > > Kafka working with two way SSL and SASL for authentication and ACL > for > > > > authorization. > > > > > > > > However, my concern with this approach was opening up Kafka brokers > > > > directly on WAN and also doing SSL termination. Is there a proxy > > > solution, > > > > where proxies live in front of Kafka brokers, so Kafka brokers are > > still > > > > hidden and proxies take care of SSL? Has anyone in the community have > > > > similar use case with Kafka, which is deployed in production? Would > > love > > > to > > > > find out your experience, feedback, or recommendation for this use > > case. > > > > > > > > Thanks in advance. > > > > > > > > PS: We are using AWS. > > > > > > > > > >
