Hope you are closing the producers. can you share the attachment through
gist/patebin
On Fri, Jan 30, 2015 at 11:11 AM, ankit tyagi <ankittyagi.mn...@gmail.com>
wrote:
> Hi Jaikiran,
>
> I am using ubuntu and was able to reproduce on redhat too. Please find the
> more information below.
>
>
> *DISTRIB_ID=Ubuntu*
> *DISTRIB_RELEASE=12.04*
> *DISTRIB_CODENAME=precise*
> *DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"*
>
> *java version "1.7.0_72"*
>
> This is happening on client side. Output of lsof was showing that maximum
> fd were FIFO and anon. But after GC FD count was reduced significantly.
>
> Below is my Client Code which i am using for publishing message.
>
>
> * private Producer<KafkaPartitionKey, KafkaEventWrapper> myProducer;*
>
> * myProducer = new Producer<>(new
> ProducerConfig(myProducerProperties));*
>
> * public void send(*
> * List<KeyedMessage<KafkaPartitionKey, KafkaEventWrapper>> msgs)*
> * {*
> * myProducer.send(msgs);*
> * }*
>
>
> we are using sync producer. I am attaching object histo before GC(histo_1)
> and after GC(histo_2) in my application.
>
> On Fri, Jan 30, 2015 at 9:34 AM, Jaikiran Pai <jai.forums2...@gmail.com>
> wrote:
>
> >
> > Which operating system are you on and what Java version? Depending on the
> > OS, you could get tools (like lsof) to show which file descriptors are
> > being held on to. Is it the client JVM which ends up with these leaks?
> >
> > Also, would it be possible to post a snippet of your application code
> > which shows how you are using the Kafka APIs?
> >
> > -Jaikiran
> > On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote:
> >
> >> Hi,
> >>
> >> Currently we are using sync producer client of 0.8.1 version in our
> >> production box . we are getting the following exception while publishing
> >> kafka message
> >>
> >> *[2015-01-29
> >> 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89]
> Fetching
> >> topic metadata with correlation id 10808 for topics [Set(*
> >> *kafka_topic_coms_FD_test1)] from broker [id:0,host:localhost,port:9092]
> >> failed*
> >> *java.net.ConnectException: Connection refused*
> >> * at sun.nio.ch.Net.connect0(Native Method)*
> >> * at sun.nio.ch.Net.connect(Net.java:465)*
> >> * at sun.nio.ch.Net.connect(Net.java:457)*
> >> * at
> >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)*
> >> at kafka.network.BlockingChannel.connect(BlockingChannel.scala:
> >> 57)
> >> at kafka.producer.SyncProducer.connect(SyncProducer.scala:141)
> >> at
> >> kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156)
> >> at
> >> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$
> >> doSend(SyncProducer.scala:68)
> >> at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
> >> at
> >> kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
> >> at
> >> kafka.producer.BrokerPartitionInfo.updateInfo(
> >> BrokerPartitionInfo.scala:82)
> >>
> >>
> >> we are using dynamic thread pool to publish message to kafka. My
> >> observation is when after keep alive time when threads in my executor
> gets
> >> destroyed, somehow file descriptor is not getting cleared but when i did
> >> explicitly ran the full gc, fd count got reduced by a signification
> amout.
> >>
> >>
> >
>
