I have shared object histogram after and before gc on gist
https://gist.github.com/ankit1987/f4a04a1350fdd609096d
On Fri, Jan 30, 2015 at 12:43 PM, Jaikiran Pai <jai.forums2...@gmail.com>
wrote:
> What kind of a (managed) component is that which has the @PreDestroy?
> Looking at the previous snippet you added, it looks like you are creating
> the Producer in some method? If you are going to close the producer in a
> @PreDestroy of the component, then you should be creating the producer in
> the @PostConstruct of the same component, so that you have proper lifecycle
> management of those resources.
>
>
> -Jaikiran
>
> On Friday 30 January 2015 12:20 PM, ankit tyagi wrote:
>
>> Hi,
>>
>> I am closing my producer at the time of shutting down my application.
>>
>> @PreDestroy
>> public void stop()
>> {
>> LOG.info("Stopping Kafka Producer for topic: {}", myTopic);
>> if (myProducer != null) {
>> myProducer.close();
>> }
>> }
>>
>>
>>
>> On Fri, Jan 30, 2015 at 11:22 AM, Manikumar Reddy <ku...@nmsworks.co.in>
>> wrote:
>>
>> Hope you are closing the producers. can you share the attachment through
>>> gist/patebin
>>>
>>> On Fri, Jan 30, 2015 at 11:11 AM, ankit tyagi <
>>> ankittyagi.mn...@gmail.com>
>>> wrote:
>>>
>>> Hi Jaikiran,
>>>>
>>>> I am using ubuntu and was able to reproduce on redhat too. Please find
>>>>
>>> the
>>>
>>>> more information below.
>>>>
>>>>
>>>> *DISTRIB_ID=Ubuntu*
>>>> *DISTRIB_RELEASE=12.04*
>>>> *DISTRIB_CODENAME=precise*
>>>> *DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"*
>>>>
>>>> *java version "1.7.0_72"*
>>>>
>>>> This is happening on client side. Output of lsof was showing that
>>>> maximum
>>>> fd were FIFO and anon. But after GC FD count was reduced significantly.
>>>>
>>>> Below is my Client Code which i am using for publishing message.
>>>>
>>>>
>>>> * private Producer<KafkaPartitionKey, KafkaEventWrapper> myProducer;*
>>>>
>>>> * myProducer = new Producer<>(new
>>>> ProducerConfig(myProducerProperties));*
>>>>
>>>> * public void send(*
>>>> * List<KeyedMessage<KafkaPartitionKey, KafkaEventWrapper>>
>>>> msgs)*
>>>> * {*
>>>> * myProducer.send(msgs);*
>>>> * }*
>>>>
>>>>
>>>> we are using sync producer. I am attaching object histo before
>>>>
>>> GC(histo_1)
>>>
>>>> and after GC(histo_2) in my application.
>>>>
>>>> On Fri, Jan 30, 2015 at 9:34 AM, Jaikiran Pai <jai.forums2...@gmail.com
>>>> >
>>>> wrote:
>>>>
>>>> Which operating system are you on and what Java version? Depending on
>>>>>
>>>> the
>>>
>>>> OS, you could get tools (like lsof) to show which file descriptors are
>>>>> being held on to. Is it the client JVM which ends up with these leaks?
>>>>>
>>>>> Also, would it be possible to post a snippet of your application code
>>>>> which shows how you are using the Kafka APIs?
>>>>>
>>>>> -Jaikiran
>>>>> On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote:
>>>>>
>>>>> Hi,
>>>>>>
>>>>>> Currently we are using sync producer client of 0.8.1 version in our
>>>>>> production box . we are getting the following exception while
>>>>>>
>>>>> publishing
>>>
>>>> kafka message
>>>>>>
>>>>>> *[2015-01-29
>>>>>> 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89]
>>>>>>
>>>>> Fetching
>>>>
>>>>> topic metadata with correlation id 10808 for topics [Set(*
>>>>>> *kafka_topic_coms_FD_test1)] from broker
>>>>>>
>>>>> [id:0,host:localhost,port:9092]
>>>
>>>> failed*
>>>>>> *java.net.ConnectException: Connection refused*
>>>>>> * at sun.nio.ch.Net.connect0(Native Method)*
>>>>>> * at sun.nio.ch.Net.connect(Net.java:465)*
>>>>>> * at sun.nio.ch.Net.connect(Net.java:457)*
>>>>>> * at
>>>>>> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)*
>>>>>> at
>>>>>>
>>>>> kafka.network.BlockingChannel.connect(BlockingChannel.scala:
>>>
>>>> 57)
>>>>>> at
>>>>>>
>>>>> kafka.producer.SyncProducer.connect(SyncProducer.scala:141)
>>>
>>>> at
>>>>>>
>>>>>> kafka.producer.SyncProducer.getOrMakeConnection(
>>> SyncProducer.scala:156)
>>>
>>>> at
>>>>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$
>>>>>> doSend(SyncProducer.scala:68)
>>>>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
>>>>>> at
>>>>>> kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
>>>>>> at
>>>>>> kafka.producer.BrokerPartitionInfo.updateInfo(
>>>>>> BrokerPartitionInfo.scala:82)
>>>>>>
>>>>>>
>>>>>> we are using dynamic thread pool to publish message to kafka. My
>>>>>> observation is when after keep alive time when threads in my executor
>>>>>>
>>>>> gets
>>>>
>>>>> destroyed, somehow file descriptor is not getting cleared but when i
>>>>>>
>>>>> did
>>>
>>>> explicitly ran the full gc, fd count got reduced by a signification
>>>>>>
>>>>> amout.
>>>>
>>>>>
>>>>>>
>
