Thanks again Rajini!
One last followup question, if you don't mind. You said that my server.properties file should look something like this: listeners=SSL://:9093 advertised.listeners=SSL://mybalancer01.example.com:9093 security.inter.broker.protocol=SSL However, please remember that I'm looking for the load balancer to terminate SSL, meaning that (my desired) communication between the load balancer and Kafka would be over plaintext (not SSL). In other words: Ruby Producers/Clients <----SSL:9093----> Load Balancer <---- Plaintext:9092 ----> Kafka So producers/client connect to the load balancer over SSL and port 9093, but then the load balancer communicates with Kafka over plaintext and port 9092. I also don't need inter broker communication to be SSL; it can be plaintext. If this is the case, do I still need to change server.properties, or can I leave it like so: listeners=plaintext://:9092 advertised.listeners=plaintext://mybalancer01.example.com:9092 Or could it just be: listeners=plaintext://:9092 advertised.listeners=plaintext://mykafka01.example.com:9092 Thanks again! Zac ________________________________ From: Rajini Sivaram <rajinisiva...@googlemail.com> Sent: Friday, November 18, 2016 9:57:22 AM To: users@kafka.apache.org Subject: Re: Can Kafka/SSL be terminated at a load balancer? You should set advertised.listeners rather than the older advertised.host.name property in server.properties: - listeners=SSL://:9093 - advertised.listeners=SSL://mybalancer01.example.com:9093 - security.inter.broker.protocol=SSL If your listeners are on particular interfaces, you can set address in the 'listeners' property too. If you want inter-broker communication to bypass the SSL proxy, you would need another security protocol that can be used for inter-broker communication (PLAINTEXT in the example below). - listeners=SSL://:9093,PLAINTEXT://:9092 - advertised.listeners=SSL://mybalancer01.example.com:9093,PLAINTEXT:// mykafka01.example.com:9092 - security.inter.broker.protocol=PLAINTEXT I haven't used the Ruby clients, so I am not sure about client configuration. With Java clients, if you don't specify truststore, the default trust stores are used, so with trusted CA-signed certificates, no additional client configuration is required. You can test your installation using the console producer and consumer that are shipped with Kafka to make sure it is working before you run with Ruby clients. On Fri, Nov 18, 2016 at 1:23 PM, Zac Harvey <zac.har...@welltok.com> wrote: > > Thanks Rajini, > > > So currently one of our Kafka nodes is 'mykafka01.example.com', and in > its server.properties file, I have advertised.host.name=mykafka01 > .example.com. Our load balancer lives at mybalancer01.example.com, and > this what producers will connect to (over SSL) to send messages to Kafka. > > > It sounds like you're saying I need to change my Kafka node's > server.properties to have advertised.host.name=mybalancer01.example.com, > yes? If not, can you perhaps provide a quick snippet of the changes I would > need to make to server.properties? > > > Again, the cert served by the balancer will be a highly-trusted (root > CA-signed) certificate that all clients will natively trust. Interestingly > enough, most (if not all) the Kafka producers/clients will be written in > Ruby (using the zendesk Kafka-Ruby gem<https://github.com/ > zendesk/ruby-kafka>), so there wont be any JKS configuration options > available for those Ruby clients. > > > Besides making the change to server.properties that I mentioned above, are > there any other client-side configs that will need to be made for the Ruby > clients to connect over SSL? > > > Thank you enormously here! > > > Best, > > Zac > > > ________________________________ > From: Rajini Sivaram <rajinisiva...@googlemail.com> > Sent: Friday, November 18, 2016 5:15:13 AM > To: users@kafka.apache.org > Subject: Re: Can Kafka/SSL be terminated at a load balancer? > > Zac, > > Kafka has its own built-in load-balancing mechanism based on partition > assignment. Requests are processed by partition leaders, distributing load > across the brokers in the cluster. If you want to put a proxy like HAProxy > with SSL termination in front of your brokers for added security, you can > do that. You can have completely independent trust chain between > clients->proxy and proxy->broker. You need to configure Kafka brokers with > the proxy host as the host in the advertised listeners for the security > protocol used by clients. > > On Thu, Nov 17, 2016 at 9:44 PM, Zac Harvey <zac.har...@welltok.com> > wrote: > > > We have two Kafka nodes and for reasons outside of this question, would > > like to set up a load balancer to terminate SSL with producers (clients). > > The SSL cert hosted by the load balancer will be signed by trusted/root > CA > > that clients should natively trust. > > > > > > Is this possible to do, or does Kafka somehow require SSL to be setup > > directly on the Kafka servers themselves? > > > > > > Thanks! > > > > > > -- > Regards, > > Rajini > -- Regards, Rajini
