[jira] [Commented] (KAFKA-1451) Broker stuck due to leader election race

[ https://issues.apache.org/jira/browse/KAFKA-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306753#comment-14306753 ] justdebugit commented on KAFKA-1451: I think the patch dose not RESOLVE the problem,it

[jira] [Updated] (KAFKA-1730) add the doc for the new java producer in 0.8.2

[ https://issues.apache.org/jira/browse/KAFKA-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1730: --- Attachment: new-producer-metrics.patch Attaching a new producer metrics/mbean doc. > add the

[jira] [Updated] (KAFKA-1922) Move doubly linked list test out of UtilsTest

[ https://issues.apache.org/jira/browse/KAFKA-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1922: Attachment: KAFKA-1922.patch > Move doubly linked list test out of UtilsTest > -

Review Request 30651: Patch for KAFKA-1922

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30651/ --- Review request for kafka. Bugs: KAFKA-1922 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1922) Move doubly linked list test out of UtilsTest

[ https://issues.apache.org/jira/browse/KAFKA-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1922: Status: Patch Available (was: Open) > Move doubly linked list test out of UtilsTest > -

[jira] [Commented] (KAFKA-1922) Move doubly linked list test out of UtilsTest

[ https://issues.apache.org/jira/browse/KAFKA-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306665#comment-14306665 ] Jiangjie Qin commented on KAFKA-1922: - Created reviewboard https://reviews.apache.org/

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306617#comment-14306617 ] Steven Zhen Wu commented on KAFKA-1919: --- this does happen with 0.8.2.0 server. In my

Re: Review Request 30063: Patch for KAFKA-1840

> On Feb. 5, 2015, 2:15 a.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/tools/MirrorMaker.scala, lines 687-689 > > > > > > Is there an expected use case for one-to-many handlers? Yes, for example some user mi

[jira] [Created] (KAFKA-1922) Move doubly linked list test out of UtilsTest

Joel Koshy created KAFKA-1922: - Summary: Move doubly linked list test out of UtilsTest Key: KAFKA-1922 URL: https://issues.apache.org/jira/browse/KAFKA-1922 Project: Kafka Issue Type: Bug

Re: Review Request 30083: Patch for KAFKA-1839

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30083/#review71133 --- Ship it! core/src/main/scala/kafka/consumer/PartitionAssignor.scal

Re: Review Request 30063: Patch for KAFKA-1840

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30063/#review71132 --- core/src/main/scala/kafka/tools/MirrorMaker.scala

Re: Review Request 29831: Patch for KAFKA-1476

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Feb. 5, 2015, 2:03 a.m.) Review request for kafka. Bugs: KAFKA-1476

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306485#comment-14306485 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-02-04_18:03:15.patch > Get a list of consumer groups > -

[jira] [Updated] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1919: - Attachment: KAFKA-1919-v1.patch > Metadata request issued with no backoff in new producer if there are no

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306357#comment-14306357 ] Jay Kreps commented on KAFKA-1919: -- Here is what I think is happening: in 0.8.1 the metad

[jira] [Commented] (KAFKA-1760) Implement new consumer client

[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306358#comment-14306358 ] Guozhang Wang commented on KAFKA-1760: -- [~jkreps], I think this is a bug, because lat

Re: Detecting lost connection in high level consumer

Thanks Guozhang and Jun for your replies. On Wed, Feb 4, 2015 at 4:06 PM, harikiran wrote: > Hey! Did anyone get a chance to read this? > > Thanks!! > > On Mon, Jan 19, 2015 at 1:26 AM harikiran wrote: > >> Hi >> >> I am using the 0811 Kafka High level consumer and I have configured " >> consum

[jira] [Commented] (KAFKA-1760) Implement new consumer client

[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306332#comment-14306332 ] Jay Kreps commented on KAFKA-1760: -- Actually shameful to say but I'm not sure this is a b

[jira] [Commented] (KAFKA-1760) Implement new consumer client

[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306324#comment-14306324 ] Guozhang Wang commented on KAFKA-1760: -- OK, the first one observed so far is that the

Re: Review Request 29831: Patch for KAFKA-1476

> On Jan. 23, 2015, 4:47 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala, line 114 > > > > > > How is this tool going to behave if the consumer's offset information > > is sto

Re: Detecting lost connection in high level consumer

Hey! Did anyone get a chance to read this? Thanks!! On Mon, Jan 19, 2015 at 1:26 AM harikiran wrote: > Hi > > I am using the 0811 Kafka High level consumer and I have configured " > consumer.timeout.ms" to a value that is not -1, say 5000ms. > > I create the consumer iterator and invoke hasNext(

Re: Review Request 29831: Patch for KAFKA-1476

> On Feb. 2, 2015, 5:42 p.m., Neha Narkhede wrote: > > Onur, we should be able to check in after these review comments are > > addressed. Also, how would deleting offsets for a group work when the > > offset storage is Kafka? It's fine to not address it in this patch. Can you > > please create

[jira] [Created] (KAFKA-1921) Delete offsets for a group with kafka offset storage

Onur Karaman created KAFKA-1921: --- Summary: Delete offsets for a group with kafka offset storage Key: KAFKA-1921 URL: https://issues.apache.org/jira/browse/KAFKA-1921 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306241#comment-14306241 ] Ashish Kumar Singh commented on KAFKA-1856: --- [~gwenshap] added checkstyle checks

Re: Review Request 29893: Patch for KAFKA-1856

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29893/ --- (Updated Feb. 4, 2015, 11:44 p.m.) Review request for kafka. Bugs: KAFKA-1856

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306237#comment-14306237 ] Ashish Kumar Singh commented on KAFKA-1856: --- Updated reviewboard https://reviews

[jira] [Updated] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated KAFKA-1856: -- Attachment: KAFKA-1856_2015-02-04_15:44:47.patch > Add PreCommit Patch Testing > ---

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306232#comment-14306232 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-02-04_15:41:50.patch > Get a list of consumer groups > -

Re: Review Request 29831: Patch for KAFKA-1476

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Feb. 4, 2015, 11:41 p.m.) Review request for kafka. Bugs: KAFKA-1476

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306178#comment-14306178 ] Ashish Kumar Singh commented on KAFKA-1856: --- [~gwenshap] so the issue was that y

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306164#comment-14306164 ] Ashish Kumar Singh commented on KAFKA-1856: --- Updated reviewboard https://reviews

[jira] [Updated] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated KAFKA-1856: -- Attachment: KAFKA-1856_2015-02-04_14:57:05.patch > Add PreCommit Patch Testing > ---

Re: Review Request 29893: Patch for KAFKA-1856

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29893/ --- (Updated Feb. 4, 2015, 10:57 p.m.) Review request for kafka. Bugs: KAFKA-1856

[jira] [Created] (KAFKA-1920) Add a metric to count client side errors in BrokerTopicMetrics

Aditya Auradkar created KAFKA-1920: -- Summary: Add a metric to count client side errors in BrokerTopicMetrics Key: KAFKA-1920 URL: https://issues.apache.org/jira/browse/KAFKA-1920 Project: Kafka

Re: Cannot stop Kafka server if zookeeper is shutdown first

Hi, KAFKA-1155 is likely Zookeeper and not the specific client. I believe the rest are already fixed in ZKClient and its a matter of asking them to release, rebase our code and make sure the issues are resolved (or that we use the features ZKClient added to resolve them). I'm a fan of Curator, bu

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305971#comment-14305971 ] Steven Zhen Wu commented on KAFKA-1919: --- this is the producer metrics when cpu shoot

[jira] [Comment Edited] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305966#comment-14305966 ] Steven Zhen Wu edited comment on KAFKA-1919 at 2/4/15 9:12 PM: -

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305966#comment-14305966 ] Steven Zhen Wu commented on KAFKA-1919: --- this is the producer metrics at the beginni

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305923#comment-14305923 ] Jakob Homan commented on KAFKA-1646: This is a big performance hit on Windows, and it

Re: kafka.utils.DoublyLinkedList

Hi Jay, The approach you suggested is exactly what we did at beginning. But Jun mentioned a case in RB that could cause problem. "Suppose there are 2 target partitions. Offset 1,2,4,5 can be routed to partition 1 and offset 3 can be routed to partition 2. When all messages are sent, you will see

[jira] [Comment Edited] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179731#comment-14179731 ] Jakob Homan edited comment on KAFKA-1646 at 2/4/15 8:36 PM: He

[jira] [Comment Edited] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231258#comment-14231258 ] Jakob Homan edited comment on KAFKA-1646 at 2/4/15 8:32 PM: Hi

[jira] [Updated] (KAFKA-1646) Improve consumer read performance for Windows

[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated KAFKA-1646: --- Assignee: xueqiang wang > Improve consumer read performance for Windows >

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

In mirror maker we do not do de-serialization on the messages. Mirror maker use source TopicPartition hash to chose a producer to send messages from the same source partition. The partition those messages end up with are decided by Partitioner class in KafkaProducer (assuming you are using the new

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305833#comment-14305833 ] Gwen Shapira commented on KAFKA-1856: - Here's what I did and the result: {code} gshap

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

Hi Jiangjie, Thanks for entertaining my question so far. Last question, I have is about serialization of message key. If the key de-serialization (Class) is not present at the MM instance, then does it use raw byte hashcode to determine the partition ? How are you going to address the situation

[jira] [Commented] (KAFKA-1730) add the doc for the new java producer in 0.8.2

[ https://issues.apache.org/jira/browse/KAFKA-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305733#comment-14305733 ] Jun Rao commented on KAFKA-1730: [~omkreddy], could you add those mbeans for the new produ

[jira] [Comment Edited] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305528#comment-14305528 ] Steven Zhen Wu edited comment on KAFKA-1919 at 2/4/15 6:11 PM: -

Re: [ANNOUNCEMENT] Apache Kafka 0.8.2.0 Released

YaY!. Thanks to Jun and everybody who contributed to this release. We have been waiting for this release for a while. Thanks, Rajasekar Elango (Salesforce.com). On Tue, Feb 3, 2015 at 8:37 PM, Jun Rao wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 0.8.

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305528#comment-14305528 ] Steven Zhen Wu commented on KAFKA-1919: --- enabled TRACE level logging. it shows tight

[jira] [Updated] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1919: - Affects Version/s: 0.8.2 > Metadata request issued with no backoff in new producer if there are no topics

[jira] [Created] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

Jay Kreps created KAFKA-1919: Summary: Metadata request issued with no backoff in new producer if there are no topics Key: KAFKA-1919 URL: https://issues.apache.org/jira/browse/KAFKA-1919 Project: Kafka

Re: Cannot stop Kafka server if zookeeper is shutdown first

<< A more abstract interface to the distributed coordination service that could be configured to use alternatives like consul or etcd would be very useful imho. +1 If we make the curator changes lets take the time to first build an interface please we can expose both meta data and async notificat

Re: kafka.utils.DoublyLinkedList

Yeah I think the Java LL is worse than that, you actually need to point at it with an iterator and call remove(). I didn't realize the scala ll was deprecated. Okay, makes sense. Would be good to move that test out of UtilsTest.scala, though. But actually, stepping back, can you guys document/expl

[Discussion] Producer Instance and Decouple Kafka Cluster State/TCP Connection Management

Hi Kafka Dev team, I would like to discuss the Kafka Cluster State Management and Producer Instance relationship in 0.8.2. Current Implementation of Producer ties very closely with Kafka Cluster, and Topic Metadata management. So imagine, you have following scenarios: Application crates m

Re: kafka.utils.DoublyLinkedList

Jay, My understanding is that java linked lists (even though they are backed by a doubly linked list) support O(1) deletion only if you have the index of the object. If you do a remove(object) it becomes O(n). As for scala lists - the scaladocs seem to indicate that the doubly linked lists are dep

Re: Cannot stop Kafka server if zookeeper is shutdown first

While on the subject of zkclient, also consider KAFKA-1793. A more abstract interface to the distributed coordination service that could be configured to use alternatives like consul or etcd would be very useful imho. Dana FWIW - the ZkClient project team have merged the pull request that I had s

Re: kafka.utils.DoublyLinkedList

Hey Jiangjie, Both the scala and java doubly linked lists allow O(1) deletion--it isn't obvious because the api is slightly different from the CS101 linked list but they do. Are you sure those wouldn't work? Can we remove this class and use those? Also the convention for unit tests is if we have

Re: Cannot stop Kafka server if zookeeper is shutdown first

curator handles sasl connection https://issues.apache.org/jira/browse/KAFKA-1695 On Wed, Feb 4, 2015, at 06:10 AM, Jaikiran Pai wrote: > FWIW - the ZkClient project team have merged the pull request that I had > submitted to allow for timeouts to operations > https://github.com/sgroschupf/zkclie

Re: Cannot stop Kafka server if zookeeper is shutdown first

FWIW - the ZkClient project team have merged the pull request that I had submitted to allow for timeouts to operations https://github.com/sgroschupf/zkclient/pull/29. I heard from Johannes (from the ZkClient project team) that they don't have any specific release date in mind but are willing to

[jira] [Created] (KAFKA-1918) System test for ZooKeeper quorum failure scenarios

Omid Aladini created KAFKA-1918: --- Summary: System test for ZooKeeper quorum failure scenarios Key: KAFKA-1918 URL: https://issues.apache.org/jira/browse/KAFKA-1918 Project: Kafka Issue Type: Te

Re: Review Request 30126: Patch for KAFKA-1845

> On Jan. 22, 2015, 7:57 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/server/KafkaConfig.scala, lines 743-758 > > > > > > This looks fairly generic. Shouldn't it be somewhere where it can be > > reused by o

Re: Review Request 30126: Patch for KAFKA-1845

> On Feb. 3, 2015, 2:06 p.m., Jeff Holoman wrote: > > core/src/main/scala/kafka/server/KafkaConfig.scala, line 834 > > > > > > Is this is the right way to do this rather than having these in > > Validators? Some of t

[jira] [Commented] (KAFKA-1913) App hungs when calls producer.send to wrong IP of Kafka broker

[ https://issues.apache.org/jira/browse/KAFKA-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304798#comment-14304798 ] Igor Khomenko commented on KAFKA-1913: -- Any ideas so far? > App hungs when calls pr

[jira] [Updated] (KAFKA-1913) App hungs when calls producer.send to wrong IP of Kafka broker

[ https://issues.apache.org/jira/browse/KAFKA-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Khomenko updated KAFKA-1913: - Environment: OS X 10.10.1, Java 7, AWS Linux (was: OS X 10.10.1, Java 7) > App hungs when calls p