[jira] [Updated] (KAFKA-1030) Addition of partitions requires bouncing all the consumers of that topic

2013-08-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-1030: - Description: Consumer may not notice new partitions because the propagation of the metadata to se

[jira] [Resolved] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-990. - Resolution: Fixed Thanks for the patches, committed v3 to 0.8 > Fix ReassignParti

[jira] [Commented] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752117#comment-13752117 ] Neha Narkhede commented on KAFKA-990: - +1 on v3. Also, it will be good to file a separa

[jira] [Commented] (KAFKA-956) High-level consumer fails to check topic metadata response for errors

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752116#comment-13752116 ] Neha Narkhede commented on KAFKA-956: - The root cause of this issue is the fact that a

[jira] [Commented] (KAFKA-1030) Addition of partitions requires bouncing all the consumers of that topic

2013-08-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752102#comment-13752102 ] Swapnil Ghike commented on KAFKA-1030: -- As Jun suggested on KAFKA-956, the easiest fi

[jira] [Created] (KAFKA-1030) Addition of partitions requires bouncing all the consumers of that topic

2013-08-27 Thread Swapnil Ghike (JIRA)
Swapnil Ghike created KAFKA-1030: Summary: Addition of partitions requires bouncing all the consumers of that topic Key: KAFKA-1030 URL: https://issues.apache.org/jira/browse/KAFKA-1030 Project: Kafka

[jira] [Closed] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jason Toffaletti (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Toffaletti closed KAFKA-1024. --- Using the default Parallel GC, memory usage hasn't grown in 4 hours and CPU usage shown by top i

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jason Toffaletti (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751846#comment-13751846 ] Jason Toffaletti commented on KAFKA-1024: - {noformat} num #instances#byt

Creating a Consumer Thread Pool

2013-08-27 Thread David Williams
Hi all, I checked out the java source and looked at the java examples. They worked well in my IDE and on the console. However, I also tried the threaded example following the consumer group example. The problem is, this example is not working and toString on the stream iterator returns the

[jira] [Comment Edited] (KAFKA-1012) Implement an Offset Manager and hook offset requests to it

2013-08-27 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751802#comment-13751802 ] Tejas Patil edited comment on KAFKA-1012 at 8/27/13 10:27 PM: --

[jira] [Commented] (KAFKA-1012) Implement an Offset Manager and hook offset requests to it

2013-08-27 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751802#comment-13751802 ] Tejas Patil commented on KAFKA-1012: Thanks a lot @guozhang for all your comments !!!

[jira] [Updated] (KAFKA-1020) Remove getAllReplicasOnBroker from KafkaController

2013-08-27 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1020: - Attachment: KAFKA-1020.v1.patch > Remove getAllReplicasOnBroker from KafkaController > --

[jira] [Updated] (KAFKA-1020) Remove getAllReplicasOnBroker from KafkaController

2013-08-27 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1020: - Status: Patch Available (was: Open) > Remove getAllReplicasOnBroker from KafkaController > -

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jason Toffaletti (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751740#comment-13751740 ] Jason Toffaletti commented on KAFKA-1024: - I may have spoken too soon, it might ju

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751723#comment-13751723 ] Jay Kreps commented on KAFKA-1024: -- Your serialized message size is 2KB, perhaps but I wo

[jira] [Resolved] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1024. -- Resolution: Not A Problem > possible memory leak in 0.8 beta1 producer with G1GC >

[jira] [Commented] (KAFKA-1023) Allow Injectable LogManager

2013-08-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751715#comment-13751715 ] Jay Kreps commented on KAFKA-1023: -- My recommendation is just to increase the retention.

[jira] [Commented] (KAFKA-1023) Allow Injectable LogManager

2013-08-27 Thread Micah Whitacre (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751705#comment-13751705 ] Micah Whitacre commented on KAFKA-1023: --- Thanks for the feedback. The fact that Log

[jira] [Commented] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

2013-08-27 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751668#comment-13751668 ] Sriram Subramanian commented on KAFKA-990: -- Guozhang - We have not merged addparti

[jira] [Commented] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

2013-08-27 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751666#comment-13751666 ] Sriram Subramanian commented on KAFKA-990: -- - made the dry run the default - added

[jira] [Updated] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

2013-08-27 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-990: - Attachment: KAFKA-990-v3.patch > Fix ReassignPartitionCommand and improve usability >

[jira] [Commented] (KAFKA-1024) possible memory leak in 0.8 beta1 producer with G1GC

2013-08-27 Thread Jason Toffaletti (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751584#comment-13751584 ] Jason Toffaletti commented on KAFKA-1024: - Process has been up for 19 hours now wi

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
Excellent Joe, thanks, good to go :) -- On 8/27/13 11:35 AM, "Joe Stein" wrote: >David, you could see that error message when the Kafka broker is not >running. > >Give the quick start a shot >http://kafka.apache.org/documentation.html#quickstart and make sure that >everything is working en

Re: Runtime problem in 8 beta1?

2013-08-27 Thread Joe Stein
David, you could see that error message when the Kafka broker is not running. Give the quick start a shot http://kafka.apache.org/documentation.html#quickstart and make sure that everything is working end to end with a producer and consumer and then hook up your code for first pass to help elimina

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
All, Sorry for this continued thread, but this is day 2 of no success making a kafka example app: I added the dependency com.yammer.metrics metrics-core 2.2.0 But this unearths yet another exception: Exception in thread "main" java.lang.reflect

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
This resulted in Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:5 7) at sun.reflect.DelegatingMethodAccessorIm

Re: Runtime problem in 8 beta1?

2013-08-27 Thread Benjamin Black
I've been using the Maven package without issue. Of course, I'm not using Spring, either. Here's the relevant section from a working pom: org.scala-lang scala-library 2.9.1 org.apache.kafka kafka_2.9.1

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
Then I added, com.101tec zkclient 0.3 But now: JarClassLoader: Warning: Null manifest from input stream associated with: lib/javax.inject-1.jar log4j:WARN No appenders could be found for logger (org.springframework.core.env.StandardEnvironment). log

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
So an update. The required version of log4j is log4j log4j 1.2.17 Joe, why is this so hard / undocumented / not caught at compile time? Now we have another error: Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAcc

[jira] [Commented] (KAFKA-1026) Dynamically Adjust Batch Size Upon Receiving MessageSizeTooLargeException

2013-08-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751519#comment-13751519 ] Swapnil Ghike commented on KAFKA-1026: -- As David Demaagd had mentioned earlier, it wi

[jira] [Commented] (KAFKA-1012) Implement an Offset Manager and hook offset requests to it

2013-08-27 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751517#comment-13751517 ] Guozhang Wang commented on KAFKA-1012: -- Thanks for the patch, happy to see some Scala

[jira] [Updated] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1029: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for patch v2. Committed

Re: kafka pull request: Escape the . in the kafka.Kafka chain

2013-08-27 Thread Jun Rao
Could you create a jira in Kafka and attach a patch there? This is to make sure that you grant permission to Apache. Thanks, Jun On Tue, Aug 27, 2013 at 9:04 AM, vpernin wrote: > GitHub user vpernin opened a pull request: > > https://github.com/apache/kafka/pull/8 > > Escape the . in

[jira] [Closed] (KAFKA-677) Retention process gives exception if an empty segment is chosen for collection

2013-08-27 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-677. - > Retention process gives exception if an empty segment is chosen for collection > ---

[jira] [Resolved] (KAFKA-677) Retention process gives exception if an empty segment is chosen for collection

2013-08-27 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-677. --- Resolution: Cannot Reproduce No longer see this. Close it for now. > Retention process gives

[jira] [Commented] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751449#comment-13751449 ] Guozhang Wang commented on KAFKA-1029: -- Thanks for v2. +1 on this one.

[jira] [Assigned] (KAFKA-1028) per topic configuration of preference for consistency over availability

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede reassigned KAFKA-1028: Assignee: Neha Narkhede > per topic configuration of preference for consistency over av

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
Ok great, thanks. What are the other required dependencies? For exmaple, log4j seems needed: Caused by: java.lang.NoClassDefFoundError: org/apache/log4j/Logger at kafka.utils.Logging$class.logger(Logging.scala:24) at kafka.utils.VerifiableProperties.logger(VerifiableProperties

[jira] [Commented] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751431#comment-13751431 ] Sam Meder commented on KAFKA-1029: -- Updated patch to also include moving the listener reg

[jira] [Updated] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1029: - Attachment: (was: 0001-KAFKA-1029-Use-brokerId-instead-of-leaderId-when-tri.patch) > Zookeeper l

[jira] [Updated] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1029: - Attachment: 0002-KAFKA-1029-Use-brokerId-instead-of-leaderId-when-tri.patch > Zookeeper leader electi

Re: Runtime problem in 8 beta1?

2013-08-27 Thread Joe Stein
2.8.0, 2.8.2, 2.9.1, 2.9.2 are all supported your POM needs a fix though, you need to match your scala-library with your kafka artifactID so your Kafka artifactID = kafka_2.9.2 so your Scala version should be 2.9.2 /*** Joe Stein Founder, Principal Con

[jira] [Commented] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751415#comment-13751415 ] Neha Narkhede commented on KAFKA-1029: -- Good catch. The election and leader change up

Re: Runtime problem in 8 beta1?

2013-08-27 Thread David Williams
Thank you for the replies. Is there a minor version in specific you recommend using? I have tried 2.8.[012] and receive a similar runtime exception: Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcJJ$sp at kafka.consumer.ConsumerConfig.(ConsumerConfig.scala:77) at com.e

[jira] [Commented] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751410#comment-13751410 ] Guozhang Wang commented on KAFKA-1029: -- Thanks for the catch Sam. Yes the comparison

kafka pull request: Escape the . in the kafka.Kafka chain

2013-08-27 Thread vpernin
GitHub user vpernin opened a pull request: https://github.com/apache/kafka/pull/8 Escape the . in the kafka.Kafka chain Little modification to the stop script to be able to stop the kill the proper process. - Escape the . in the kafka.Kafka chain - Also add a grep java to ge

[jira] [Updated] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1029: - Attachment: 0001-KAFKA-1029-Use-brokerId-instead-of-leaderId-when-tri.patch > Zookeeper leader electi

[jira] [Updated] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1029: - Status: Patch Available (was: Open) > Zookeeper leader election stuck in ephemeral node retry loop >

[jira] [Created] (KAFKA-1029) Zookeeper leader election stuck in ephemeral node retry loop

2013-08-27 Thread Sam Meder (JIRA)
Sam Meder created KAFKA-1029: Summary: Zookeeper leader election stuck in ephemeral node retry loop Key: KAFKA-1029 URL: https://issues.apache.org/jira/browse/KAFKA-1029 Project: Kafka Issue Typ