Metadata Update Corner Case

While updating a small Kafka client application to use Kafka 1.1.1 and the KafkaProducer (rather than the deprecated Producer) a maintenance related unit test started failing. The scenario is out of the ordinary, but, the resulting problem may still be of interest: * Two Kafka nodes * O

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702992#comment-14702992 ] James Lent commented on KAFKA-1387: --- Your approach sounds much simpler than mine (w

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14694052#comment-14694052 ] James Lent commented on KAFKA-1387: --- After refreshing my memory of this issue I

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692595#comment-14692595 ] James Lent commented on KAFKA-1387: --- It has been a while since I investigated

Re: ZooKeeper Session Timeout During Startup Caused By Long ParNewGC Pauses

527797K(4160256K), 21.8446524 secs] [Times: user=87.31 sys=0.01, real=21.84 secs] 491.085: [GC (Allocation Failure) 491.085: [ParNew James. Thanks for reporting this. So the issue only shows up at Kafka startup time? After that, the GC time is back to normal again? Jun On Tue, Apr 28, 2015

Re: ZooKeeper Session Timeout During Startup Caused By Long ParNewGC Pauses

=0.01, real=9.15 secs] [2015-04-24 13:26:33,443] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener) James Lent Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com/> jl...@digitalsmiths.com<mailto:jl

Re: ZooKeeper Session Timeout During Startup Caused By Long ParNewGC Pauses

Here is what the ParNewGC pauses look like in the Kafka logs in one of our environments (earlier I provided logs from the test program). In this case I had already increased the ZooKeeper session timeout to 20 seconds so that it did not trigger a communication failure while waiting for GC to fi

ZooKeeper Session Timeout During Startup Caused By Long ParNewGC Pauses

ation Failure) 0.625: [ParNew: 300257K->21936K(306688K), 0.0368052 secs] 842041K->584850K(4160256K), 0.0368367 secs] [Times: user=0.08 sys=0.00, real=0.03 secs] ... Playing with the other two parameters can vary the results, but, they are all bad if the size of the static object is in the &q

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156746#comment-14156746 ] James Lent commented on KAFKA-1387: --- Good idea and done: http://

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153370#comment-14153370 ] James Lent commented on KAFKA-1387: --- I have messed things up. I tried to use

[jira] [Updated] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Lent updated KAFKA-1387: -- Attachment: kafka-1387.patch > Kafka getting stuck creating ephemeral node it has already created w

[jira] [Updated] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Lent updated KAFKA-1387: -- Labels: newbie patch (was: ) Affects Version/s: 0.8.1.1 Status: Patch

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152453#comment-14152453 ] James Lent commented on KAFKA-1387: --- As for your question (which I agree is one of

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152337#comment-14152337 ] James Lent commented on KAFKA-1387: --- I aplogize in advance for my ignorance, bu

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151673#comment-14151673 ] James Lent edited comment on KAFKA-1387 at 9/29/14 2:4

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151673#comment-14151673 ] James Lent edited comment on KAFKA-1387 at 9/29/14 2:1

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151663#comment-14151663 ] James Lent edited comment on KAFKA-1387 at 9/29/14 1:4

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151673#comment-14151673 ] James Lent edited comment on KAFKA-1387 at 9/29/14 1:2

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151673#comment-14151673 ] James Lent commented on KAFKA-1387: --- In case anyone is interested in the complete

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151663#comment-14151663 ] James Lent commented on KAFKA-1387: --- As background we are using ZooKeeper 3.4.5.

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 4:08 PM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 4:05 PM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 4:00 PM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 3:55 PM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 3:05 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 3:05 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 3:02 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 2:57 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 2:50 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 2:51 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 2:50 AM: ---

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent edited comment on KAFKA-1387 at 9/28/14 2:47 AM: ---

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150883#comment-14150883 ] James Lent commented on KAFKA-1387: --- I have seen this issue in our QA environmen