Re: [VOTE] 0.10.0.0 RC6

2016-05-21 Thread Neha Narkhede
+1 (binding). Verified the quickstart. On Sat, May 21, 2016 at 9:22 AM, Ismael Juma wrote: > +1 (non-binding). > > Verified source and binary artifacts, ran ./gradlew testAll with JDK 7u80, > quick start on source artifact and Scala 2.11 binary artifact. And the > system tests passed on the 0.10

Re: [DISCUSS] scalability limits in the coordinator

2016-05-21 Thread Onur Karaman
Results without compression: 1 consumer 292383 bytes 5 consumers 1079579 bytes * the tipping point 10 consumers 1855018 bytes 20 consumers 2780220 bytes 30 consumers 3705422 bytes 40 consumers 4630624 bytes 50 consumers 826 bytes 60 consumers 6480788 bytes 70 consumers 7405750 bytes 80 consumer

Re: [DISCUSS] scalability limits in the coordinator

2016-05-21 Thread Jun Rao
Onur, Thanks for the investigation. Another option is to just fix how we deal with the case when a message is larger than the fetch size. Today, if the fetch size is smaller than the fetch size, the consumer will get stuck. Instead, we can simply return the full message if it's larger than the fe

Re: [VOTE] 0.10.0.0 RC6

2016-05-21 Thread Ismael Juma
+1 (non-binding). Verified source and binary artifacts, ran ./gradlew testAll with JDK 7u80, quick start on source artifact and Scala 2.11 binary artifact. And the system tests passed on the 0.10.0 branch: https://jenkins.confluent.io/job/system-test-kafka-0.10.0/61/ Ismael On Sat, May 21, 2016

[jira] [Commented] (KAFKA-3632) ConsumerLag metrics persist after partition migration

2016-05-21 Thread Otis Gospodnetic (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295113#comment-15295113 ] Otis Gospodnetic commented on KAFKA-3632: - Wow, could this be a fix for the proble

[GitHub] kafka-site pull request: KAFKA-3709: Create a project security pag...

2016-05-21 Thread junrao
Github user junrao commented on the pull request: https://github.com/apache/kafka-site/pull/12#issuecomment-220784809 The mailing list has been created. We can invite all committers/contributors who are interested in security to join the list. secur...@kafka.apache.org

[jira] [Commented] (KAFKA-3709) Create project security page

2016-05-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295087#comment-15295087 ] ASF GitHub Bot commented on KAFKA-3709: --- Github user junrao commented on the pull re

[RESULTS] [VOTE] Release Kafka version 0.10.0.0

2016-05-21 Thread Gwen Shapira
This vote passes with 9 +1 votes (4 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jay Kreps * Jun Rao * Guozhang Wang * Joe Stein Committers: * Sriharsha Chintalapani * Ewen Cheslack-Postava Community: * Dana Powers * Vahid S. Hashemian * Ashish Singh Vote thread:https://www.mail-arc

[DISCUSS] scalability limits in the coordinator

2016-05-21 Thread Onur Karaman
Hey everyone. So I started doing some tests on the new consumer/coordinator to see if it could handle more strenuous use cases like mirroring clusters with thousands of topics and thought I'd share whatever I have so far. The scalability limit: the amount of group metadata we can fit into one mess

Re: Apache Kafka JIRA Worflow: Add Closed -> Reopen transition

2016-05-21 Thread Ismael Juma
Thanks Manikumar, I had previously tried to fix the issue below but couldn't because JIRA did not allow the issue to be reopened. So, I think this is a good change. We can revisit if it becomes an issue. And thanks for all your help cleaning up JIRA (both via closing stale issues as well as fixing