Hi, I have setup SSL (port 9093) using keystore / truststore on each broker and as you can see, it works if I specify the truststore, doesn’t work if I don’t:
root@8681fd9da149:/test# kafka-console-producer --broker-list localhost:9093 --topic test_ssl hi [2016-12-21 04:09:16,527] WARN Bootstrap broker localhost:9093 disconnected (org.apache.kafka.clients.NetworkClient) [2016-12-21 04:09:16,678] WARN Bootstrap broker localhost:9093 disconnected (org.apache.kafka.clients.NetworkClient) [2016-12-21 04:09:16,842] WARN Bootstrap broker localhost:9093 disconnected (org.apache.kafka.clients.NetworkClient) root@8681fd9da149:/test# kafka-console-producer --broker-list localhost:9093 --topic test_ssl --producer.config client-ssl.properties hello world ^Croot@8681fd9da149:/test# That makes me believe that SSL is setup correctly. Currently security.inter.broker.protocol is set to PLAINTEXT, but as soon as I set it to SSL on one broker, I get the following error: [2016-12-21 04:04:20,093] WARN [ReplicaFetcherThread-0-6], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@7f6b8a8 (kafka.server.ReplicaFetcherThread) java.io.IOException: Connection to broker-6.example.com.au:9093 (id: 6 rack: null) failed at kafka.utils.NetworkClientBlockingOps$.awaitReady$1(NetworkClientBlockingOps.scala:83) at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:93) at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:248) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:238) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) Basically the broker I just switched can’t connect to any other broker. Telnet on that hostname and port works, so it’s reachable. Do you know what happening and how I can fix it? My Kafka cluster starts with the following config: security.inter.broker.protocol = SSL ssl.cipher.suites = null ssl.client.auth = none ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = HTTPS ssl.key.password = [hidden] ssl.keymanager.algorithm = SunX509 ssl.keystore.location = /etc/kafka/secrets/server.keystore.jks ssl.keystore.password = [hidden] ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = SHA1PRNG ssl.trustmanager.algorithm = PKIX ssl.truststore.location = /etc/kafka/secrets/truststore.jks ssl.truststore.password = [hidden] ssl.truststore.type = JKS I may be missing something obvious here? Kind regards, Stephane
