Hi,
I am closing my producer at the time of shutting down my application.
@PreDestroy
public void stop()
{
LOG.info("Stopping Kafka Producer for topic: {}", myTopic);
if (myProducer != null) {
myProducer.close();
}
}
On Fri, Jan 30, 2015 at 11:22 AM, Manikumar Reddy <ku...@nmsworks.co.in>
wrote:
> Hope you are closing the producers. can you share the attachment through
> gist/patebin
>
> On Fri, Jan 30, 2015 at 11:11 AM, ankit tyagi <ankittyagi.mn...@gmail.com>
> wrote:
>
> > Hi Jaikiran,
> >
> > I am using ubuntu and was able to reproduce on redhat too. Please find
> the
> > more information below.
> >
> >
> > *DISTRIB_ID=Ubuntu*
> > *DISTRIB_RELEASE=12.04*
> > *DISTRIB_CODENAME=precise*
> > *DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"*
> >
> > *java version "1.7.0_72"*
> >
> > This is happening on client side. Output of lsof was showing that maximum
> > fd were FIFO and anon. But after GC FD count was reduced significantly.
> >
> > Below is my Client Code which i am using for publishing message.
> >
> >
> > * private Producer<KafkaPartitionKey, KafkaEventWrapper> myProducer;*
> >
> > * myProducer = new Producer<>(new
> > ProducerConfig(myProducerProperties));*
> >
> > * public void send(*
> > * List<KeyedMessage<KafkaPartitionKey, KafkaEventWrapper>> msgs)*
> > * {*
> > * myProducer.send(msgs);*
> > * }*
> >
> >
> > we are using sync producer. I am attaching object histo before
> GC(histo_1)
> > and after GC(histo_2) in my application.
> >
> > On Fri, Jan 30, 2015 at 9:34 AM, Jaikiran Pai <jai.forums2...@gmail.com>
> > wrote:
> >
> > >
> > > Which operating system are you on and what Java version? Depending on
> the
> > > OS, you could get tools (like lsof) to show which file descriptors are
> > > being held on to. Is it the client JVM which ends up with these leaks?
> > >
> > > Also, would it be possible to post a snippet of your application code
> > > which shows how you are using the Kafka APIs?
> > >
> > > -Jaikiran
> > > On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote:
> > >
> > >> Hi,
> > >>
> > >> Currently we are using sync producer client of 0.8.1 version in our
> > >> production box . we are getting the following exception while
> publishing
> > >> kafka message
> > >>
> > >> *[2015-01-29
> > >> 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89]
> > Fetching
> > >> topic metadata with correlation id 10808 for topics [Set(*
> > >> *kafka_topic_coms_FD_test1)] from broker
> [id:0,host:localhost,port:9092]
> > >> failed*
> > >> *java.net.ConnectException: Connection refused*
> > >> * at sun.nio.ch.Net.connect0(Native Method)*
> > >> * at sun.nio.ch.Net.connect(Net.java:465)*
> > >> * at sun.nio.ch.Net.connect(Net.java:457)*
> > >> * at
> > >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)*
> > >> at
> kafka.network.BlockingChannel.connect(BlockingChannel.scala:
> > >> 57)
> > >> at
> kafka.producer.SyncProducer.connect(SyncProducer.scala:141)
> > >> at
> > >>
> kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156)
> > >> at
> > >> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$
> > >> doSend(SyncProducer.scala:68)
> > >> at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
> > >> at
> > >> kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
> > >> at
> > >> kafka.producer.BrokerPartitionInfo.updateInfo(
> > >> BrokerPartitionInfo.scala:82)
> > >>
> > >>
> > >> we are using dynamic thread pool to publish message to kafka. My
> > >> observation is when after keep alive time when threads in my executor
> > gets
> > >> destroyed, somehow file descriptor is not getting cleared but when i
> did
> > >> explicitly ran the full gc, fd count got reduced by a signification
> > amout.
> > >>
> > >>
> > >
> >
>
