Re: Exposing Kafka on WAN

2018-08-31 Thread Robin Moffatt
Shameless plug for an article I wrote on understanding Kafka listeners better, which might help in this case: https://rmoff.net/2018/08/02/kafka-listeners-explained/ On Thu, 30 Aug 2018 at 14:14, Andrew Otto wrote: > The trouble is that the producer and consumer clients need to discover the >

Re: Exposing Kafka on WAN

2018-08-30 Thread Andrew Otto
The trouble is that the producer and consumer clients need to discover the broker hostnames and address the individual brokers directly. There is an advertised.listeners setting that will allow you to tell clients to connect to external proxy hostnames instead of your internal ones, but those prox

Re: Exposing Kafka on WAN

2018-08-30 Thread Dan Markhasin
Usually for such a use case you'd have a physical load balancer box (F5, etc.) in front of Kafka that would handle the SSL termination, but it should be possible with NGINX as well: https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-tcp/ On Fri, 24 Aug 2018 at 18:35, Jack

Re: Exposing Kafka on WAN

2018-08-24 Thread Jack S
Thanks Ryanne. That's one of the options we had considered. I was hoping to keep solution simple and efficient. With HTTP proxy, we would have to worry about configurations, scalability, and operation. This is probably true with proxy solution as well, but at least my thinking was that deploying p

Re: Exposing Kafka on WAN

2018-08-24 Thread Ryanne Dolan
Can you use a Kafka HTTP proxy instead of using the Kafka protocol directly? Ryanne On Thu, Aug 23, 2018, 7:29 PM Jack S wrote: > Hello, > > We have a requirement for opening Kafka on WAN where external producers and > consumers need to be able to talk to Kafka. I was able to get Zookeeper and