Hi, Currently we are using sync producer client of 0.8.1 version in our production box . we are getting the following exception while publishing kafka message
*[2015-01-29 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89] Fetching topic metadata with correlation id 10808 for topics [Set(* *kafka_topic_coms_FD_test1)] from broker [id:0,host:localhost,port:9092] failed* *java.net.ConnectException: Connection refused* * at sun.nio.ch.Net.connect0(Native Method)* * at sun.nio.ch.Net.connect(Net.java:465)* * at sun.nio.ch.Net.connect(Net.java:457)* * at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)* at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) at kafka.producer.SyncProducer.connect(SyncProducer.scala:141) at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) at kafka.producer.SyncProducer.send(SyncProducer.scala:112) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) we are using dynamic thread pool to publish message to kafka. My observation is when after keep alive time when threads in my executor gets destroyed, somehow file descriptor is not getting cleared but when i did explicitly ran the full gc, fd count got reduced by a signification amout.