Which operating system are you on and what Java version? Depending on
the OS, you could get tools (like lsof) to show which file descriptors
are being held on to. Is it the client JVM which ends up with these leaks?
Also, would it be possible to post a snippet of your application code
which shows how you are using the Kafka APIs?
-Jaikiran
On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote:
Hi,
Currently we are using sync producer client of 0.8.1 version in our
production box . we are getting the following exception while publishing
kafka message
*[2015-01-29
13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89] Fetching
topic metadata with correlation id 10808 for topics [Set(*
*kafka_topic_coms_FD_test1)] from broker [id:0,host:localhost,port:9092]
failed*
*java.net.ConnectException: Connection refused*
* at sun.nio.ch.Net.connect0(Native Method)*
* at sun.nio.ch.Net.connect(Net.java:465)*
* at sun.nio.ch.Net.connect(Net.java:457)*
* at
sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)*
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:141)
at
kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156)
at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
at
kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
at
kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
we are using dynamic thread pool to publish message to kafka. My
observation is when after keep alive time when threads in my executor gets
destroyed, somehow file descriptor is not getting cleared but when i did
explicitly ran the full gc, fd count got reduced by a signification amout.
