Shameless plug for an article I wrote on understanding Kafka listeners better, which might help in this case: https://rmoff.net/2018/08/02/kafka-listeners-explained/
On Thu, 30 Aug 2018 at 14:14, Andrew Otto <o...@wikimedia.org> wrote: > The trouble is that the producer and consumer clients need to discover the > broker hostnames and address the individual brokers directly. There is an > advertised.listeners setting that will allow you to tell clients to connect > to external proxy hostnames instead of your internal ones, but those > proxies will need to know how to map directly from an advertised hostname > to an internal kafka broker hostname. You’ll need some logic in your proxy > to do that routing. > > P.S. I’ve not actually set this up before, but this is my understanding :) > > > > On Thu, Aug 30, 2018 at 7:16 AM Dan Markhasin <minimi...@gmail.com> wrote: > > > Usually for such a use case you'd have a physical load balancer box (F5, > > etc.) in front of Kafka that would handle the SSL termination, but it > > should be possible with NGINX as well: > > > > > > > https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-tcp/ > > > > On Fri, 24 Aug 2018 at 18:35, Jack S <jacksilver1...@gmail.com> wrote: > > > > > Thanks Ryanne. > > > > > > That's one of the options we had considered. I was hoping to keep > > solution > > > simple and efficient. With HTTP proxy, we would have to worry about > > > configurations, scalability, and operation. This is probably true with > > > proxy solution as well, but at least my thinking was that deploying > proxy > > > would be more standard with less management effort on our side. Also, > we > > > are very familiar with Kafka usual producer/consumer usage, operation, > > etc. > > > and could re-use much of our producer and consumer infrastructure that > we > > > currently use internally. > > > > > > Having said that, this is where I was hoping to hear and get feedback > > from > > > community - what people have deployed with such use case and any > > learnings > > > and suggestions. > > > > > > On Fri, Aug 24, 2018 at 7:42 AM Ryanne Dolan <ryannedo...@gmail.com> > > > wrote: > > > > > > > Can you use a Kafka HTTP proxy instead of using the Kafka protocol > > > > directly? > > > > > > > > Ryanne > > > > > > > > On Thu, Aug 23, 2018, 7:29 PM Jack S <jacksilver1...@gmail.com> > wrote: > > > > > > > > > Hello, > > > > > > > > > > We have a requirement for opening Kafka on WAN where external > > producers > > > > and > > > > > consumers need to be able to talk to Kafka. I was able to get > > Zookeeper > > > > and > > > > > Kafka working with two way SSL and SASL for authentication and ACL > > for > > > > > authorization. > > > > > > > > > > However, my concern with this approach was opening up Kafka brokers > > > > > directly on WAN and also doing SSL termination. Is there a proxy > > > > solution, > > > > > where proxies live in front of Kafka brokers, so Kafka brokers are > > > still > > > > > hidden and proxies take care of SSL? Has anyone in the community > have > > > > > similar use case with Kafka, which is deployed in production? Would > > > love > > > > to > > > > > find out your experience, feedback, or recommendation for this use > > > case. > > > > > > > > > > Thanks in advance. > > > > > > > > > > PS: We are using AWS. > > > > > > > > > > > > > > >
