Hi, Is it possible this happens because your producer can't establish a secure connection to the Kafka brokers but repeatedly tries to? Do you see any SSL errors in the kafka broker logs or in the logs for your producer?
Harper On Fri, Oct 18, 2019 at 8:23 AM DHARSHAN SHAS3 <dharshanshas...@gmail.com> wrote: > Hi , > > > > > > I request you all to help me understand why enabling SSL on Kafka nodes > results in increased number of TCP TIME_WAIT connections on Kafka brokers. > Recently, I enabled SSL on > > Kafka broker and also enabled SSL on producer (Spring Application) and what > I see is SSL works fine as expected but Producer keeps opening and closing > connection with brokers which is > > contributing to CPU hike on application side. > > > > Couple of questions > > > > 1. Is enabling SSL expected to increase the no of connections to Kafka > broker? > > 2. Is there a specific config at producer/broker level to address this? > > 3. Why does a producer keep opening/closing connections? > > > > Here's a sample of netstat command connection statistics on Kafka broker > > > > Type of connections : > > 31 ESTABLISHED > > 1 FIN_WAIT1 > > 1 FIN_WAIT2 > > 3 LISTEN > > 1997 TIME_WAIT > > > > > > Spring Producer connection settings > > > > kafka.producer.batch.size=16384 > > kafka.producer.bootstrap.servers=localhost:9093 > > kafka.producer.buffer.memory=33554432 > > > kafka.producer.key.serializer.class=org.apache.kafka.common.serialization.StringSerializer > > kafka.producer.linger.ms=1 > > kafka.producer.retries=0 > > kafka.producer.value.serializer.class=com.org.KafkaJsonSerializer > > kafka.producer.topic.audit=Audit > > kafka.producer.topic.audit.test=audit-trail-test > > kafka.producer.topic.crl=certificate-revocation > > kafka.test.to.test.topic.t=Aer > > kafka.producer.topic.data=compacted > > kafka.producer.topic.log=log > > > > ssl.keystore.location=/test.com/data/test/ssl/keystore/kafka.keystore.jks > > ssl.truststore.location=/ > test.com/data/kafka/ssl/truststore/kafka.truststore.jks > > ssl.key.password=** > > ssl.keystore.password=** > > ssl.truststore.password=** > > security.protocol=SSL > > ssl.protocol=TLS > > ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 > > ssl.keystore.type=JKS > > ssl.truststore.type=JKS >
