Hi, I am closing my producer at the time of shutting down my application.
@PreDestroy public void stop() { LOG.info("Stopping Kafka Producer for topic: {}", myTopic); if (myProducer != null) { myProducer.close(); } } On Fri, Jan 30, 2015 at 11:22 AM, Manikumar Reddy <ku...@nmsworks.co.in> wrote: > Hope you are closing the producers. can you share the attachment through > gist/patebin > > On Fri, Jan 30, 2015 at 11:11 AM, ankit tyagi <ankittyagi.mn...@gmail.com> > wrote: > > > Hi Jaikiran, > > > > I am using ubuntu and was able to reproduce on redhat too. Please find > the > > more information below. > > > > > > *DISTRIB_ID=Ubuntu* > > *DISTRIB_RELEASE=12.04* > > *DISTRIB_CODENAME=precise* > > *DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"* > > > > *java version "1.7.0_72"* > > > > This is happening on client side. Output of lsof was showing that maximum > > fd were FIFO and anon. But after GC FD count was reduced significantly. > > > > Below is my Client Code which i am using for publishing message. > > > > > > * private Producer<KafkaPartitionKey, KafkaEventWrapper> myProducer;* > > > > * myProducer = new Producer<>(new > > ProducerConfig(myProducerProperties));* > > > > * public void send(* > > * List<KeyedMessage<KafkaPartitionKey, KafkaEventWrapper>> msgs)* > > * {* > > * myProducer.send(msgs);* > > * }* > > > > > > we are using sync producer. I am attaching object histo before > GC(histo_1) > > and after GC(histo_2) in my application. > > > > On Fri, Jan 30, 2015 at 9:34 AM, Jaikiran Pai <jai.forums2...@gmail.com> > > wrote: > > > > > > > > Which operating system are you on and what Java version? Depending on > the > > > OS, you could get tools (like lsof) to show which file descriptors are > > > being held on to. Is it the client JVM which ends up with these leaks? > > > > > > Also, would it be possible to post a snippet of your application code > > > which shows how you are using the Kafka APIs? > > > > > > -Jaikiran > > > On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote: > > > > > >> Hi, > > >> > > >> Currently we are using sync producer client of 0.8.1 version in our > > >> production box . we are getting the following exception while > publishing > > >> kafka message > > >> > > >> *[2015-01-29 > > >> 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89] > > Fetching > > >> topic metadata with correlation id 10808 for topics [Set(* > > >> *kafka_topic_coms_FD_test1)] from broker > [id:0,host:localhost,port:9092] > > >> failed* > > >> *java.net.ConnectException: Connection refused* > > >> * at sun.nio.ch.Net.connect0(Native Method)* > > >> * at sun.nio.ch.Net.connect(Net.java:465)* > > >> * at sun.nio.ch.Net.connect(Net.java:457)* > > >> * at > > >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)* > > >> at > kafka.network.BlockingChannel.connect(BlockingChannel.scala: > > >> 57) > > >> at > kafka.producer.SyncProducer.connect(SyncProducer.scala:141) > > >> at > > >> > kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156) > > >> at > > >> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$ > > >> doSend(SyncProducer.scala:68) > > >> at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > > >> at > > >> kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > > >> at > > >> kafka.producer.BrokerPartitionInfo.updateInfo( > > >> BrokerPartitionInfo.scala:82) > > >> > > >> > > >> we are using dynamic thread pool to publish message to kafka. My > > >> observation is when after keep alive time when threads in my executor > > gets > > >> destroyed, somehow file descriptor is not getting cleared but when i > did > > >> explicitly ran the full gc, fd count got reduced by a signification > > amout. > > >> > > >> > > > > > >