Re: Review Request 36578: Patch for KAFKA-2338

2015-07-20 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36578/#review92374 --- In addition to inline comments, the patch fell out of sync recently

[jira] [Commented] (KAFKA-1621) Standardize --messages option in perf scripts

2015-07-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634555#comment-14634555 ] ASF GitHub Bot commented on KAFKA-1621: --- Github user rekhajoshm closed the pull requ

[GitHub] kafka pull request: KAFKA-1621 : Standardize --messages option

2015-07-20 Thread rekhajoshm
Github user rekhajoshm closed the pull request at: https://github.com/apache/kafka/pull/58 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Created] (KAFKA-2351) Brokers are having a problem shutting down correctly

2015-07-20 Thread Mayuresh Gharat (JIRA)
Mayuresh Gharat created KAFKA-2351: -- Summary: Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue

[jira] [Commented] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

2015-07-20 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634465#comment-14634465 ] Edward Ribeiro commented on KAFKA-2338: --- Hi [~gwenshap], whenever you have a free cy

[jira] [Commented] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634435#comment-14634435 ] Gwen Shapira commented on KAFKA-2100: - Go ahead, all yours :) > Client Error doesn't

[jira] [Updated] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2100: Assignee: David Jacot (was: Gwen Shapira) > Client Error doesn't preserve or display original serve

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
> On May 22, 2015, 12:33 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > line 294 > > > > > > Not sure why we are looping back here. If the HandshakeStatus is

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
> On June 24, 2015, 4:39 p.m., Jiangjie Qin wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 75-78 > > > > > > Can they be replaced by clear() method? These are set o

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
> On June 30, 2015, 5:20 p.m., Joel Koshy wrote: > > clients/src/main/java/org/apache/kafka/common/network/Channel.java, lines > > 148-150 > > > > > > This should be caught by checkstyle - can you double-check? If we

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
> On June 30, 2015, 3:03 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 393 > > > > > > This parameter doesn't seem to be used. We can remove it. I am using for metrics > On Jun

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
> On June 30, 2015, 3:03 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > line 67 > > > > > > Shouldn't we only start the ssl handshake after the raw socket >

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92361 --- core/src/main/scala/kafka/security/auth/PermissionType.scala (line

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92359 --- core/src/main/scala/kafka/security/auth/PermissionType.scala (line

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92358 --- core/src/main/scala/kafka/server/KafkaApis.scala (line 163)

[jira] [Commented] (KAFKA-2275) Add a ListTopics() API to the new consumer

2015-07-20 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634404#comment-14634404 ] Ashish K Singh commented on KAFKA-2275: --- [~hachikuji] thanks for your response. I wa

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92357 --- core/src/main/scala/kafka/server/KafkaApis.scala (line 136)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92356 --- core/src/main/scala/kafka/server/KafkaApis.scala (line 536)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92355 --- core/src/main/scala/kafka/server/KafkaApis.scala (line 148)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92354 --- core/src/main/scala/kafka/security/auth/Operation.scala (line 43) <

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92353 --- core/src/main/scala/kafka/security/auth/KafkaPrincipal.scala (line

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92352 --- core/src/main/scala/kafka/security/auth/KafkaPrincipal.scala (line

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92351 --- core/src/main/scala/kafka/security/auth/Acl.scala (line 86)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92350 --- core/src/main/scala/kafka/server/KafkaApis.scala (line 620)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92349 --- core/src/main/scala/kafka/security/auth/Acl.scala (line 136)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92347 --- core/src/main/scala/kafka/security/auth/PermissionType.scala (line

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92345 --- core/src/main/scala/kafka/security/auth/Acl.scala (line 71)

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review92341 --- core/src/main/scala/kafka/security/auth/Acl.scala (line 86)

Build failed in Jenkins: KafkaPreCommit #159

2015-07-20 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-2328; merge-kafka-pr.py script should not leave user in a detached branch -- [...truncated 803 lines...] kafka.log.OffsetMapTest > testClear PASSED kafka.log.LogManage

[jira] [Updated] (KAFKA-2337) Verify that metric names will not collide when creating new topics

2015-07-20 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2337: --- Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) > Ve

Jenkins build is back to normal : Kafka-trunk #550

2015-07-20 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-824) java.lang.NullPointerException in commitOffsets

2015-07-20 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634327#comment-14634327 ] Parth Brahmbhatt commented on KAFKA-824: [~junrao] Looked at the fix code and ensur

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634322#comment-14634322 ] Yasuhiro Matsuda commented on KAFKA-2350: - To me the state preservation is not a r

Build failed in Jenkins: KafkaPreCommit #158

2015-07-20 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-2337; Verify that metric names will not collide when creating new topics; patched by Grant Henke; reviewed by Edward Ribeiro, Ashish Singh and Gwen Shapira -- [...tru

[jira] [Comment Edited] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634294#comment-14634294 ] Jason Gustafson edited comment on KAFKA-2350 at 7/20/15 11:49 PM: --

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634296#comment-14634296 ] Edward Ribeiro commented on KAFKA-2350: --- Nit: wouldn't the inverse of ``pause`` to b

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634294#comment-14634294 ] Jason Gustafson commented on KAFKA-2350: I don't see why not. In that case, I assu

[jira] [Updated] (KAFKA-2328) merge-kafka-pr.py script should not leave user in a detached branch

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2328: Resolution: Fixed Reviewer: Gwen Shapira Fix Version/s: 0.8.3 Status: Reso

[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-07-20 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Status: Patch Available (was: In Progress) > KafkaAuthorizer: Add all public entities, conf

[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-07-20 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-07-20_16:42:18.patch > KafkaAuthorizer: Add all public entities,

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Parth Brahmbhatt
> On July 16, 2015, 10:32 p.m., Edward Ribeiro wrote: > > core/src/main/scala/kafka/security/auth/Operation.scala, lines 43-44 > > > > > > +1. As of today, it spills out a Match error that can be confusing to > > fig

[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-07-20 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634280#comment-14634280 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apa

[jira] [Commented] (KAFKA-2328) merge-kafka-pr.py script should not leave user in a detached branch

2015-07-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634279#comment-14634279 ] ASF GitHub Bot commented on KAFKA-2328: --- Github user asfgit closed the pull request

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Parth Brahmbhatt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/ --- (Updated July 20, 2015, 11:42 p.m.) Review request for kafka. Bugs: KAFKA-221

[GitHub] kafka pull request: KAFKA-2328; merge-kafka-pr.py script should no...

2015-07-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/84 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

Re: Review Request 34492: Patch for KAFKA-2210

2015-07-20 Thread Parth Brahmbhatt
> On July 16, 2015, 2:16 a.m., Jun Rao wrote: > > Thanks for the patch. A few comments below. > > > > Also, > > 1. For making user/group case-insensitive, could you start a discussion > > thread on the dev mailing list to see if anyone has concerns on this? > > 2. Got the following compilation

[GitHub] kafka pull request: Fixed javadoc for committed return value

2015-07-20 Thread sslavic
GitHub user sslavic opened a pull request: https://github.com/apache/kafka/pull/89 Fixed javadoc for committed return value If no offset has been committed, then committed method does not return (null) value, instead NoOffsetForPartitionException is thrown in that case. You can mer

[jira] [Commented] (KAFKA-2342) transient unit test failure in testConsumptionWithBrokerFailures

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634274#comment-14634274 ] Jason Gustafson commented on KAFKA-2342: [~junrao], I found a case where a rebalan

[jira] [Commented] (KAFKA-2342) transient unit test failure in testConsumptionWithBrokerFailures

2015-07-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634266#comment-14634266 ] ASF GitHub Bot commented on KAFKA-2342: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request: KAFKA-2342; fix transient unit test failure Co...

2015-07-20 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/88 KAFKA-2342; fix transient unit test failure ConsumerBounceTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-2342 Alter

[jira] [Commented] (KAFKA-2337) Verify that metric names will not collide when creating new topics

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634247#comment-14634247 ] Gwen Shapira commented on KAFKA-2337: - Pushed to trunk. Thanks for the patch [~granth

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634229#comment-14634229 ] Yasuhiro Matsuda commented on KAFKA-2350: - Can we have TopicPartition rather than

[jira] [Commented] (KAFKA-2329) Consumers balance fails when multiple consumers are started simultaneously.

2015-07-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634222#comment-14634222 ] Guozhang Wang commented on KAFKA-2329: -- Thanks for the patch [~zklapow], a couple com

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Ben Kirwin
Yes, sorry, I think this is right -- it's pretty application-specific. One thing to note: in a large subset of cases (ie. bulk load, copycat-type, mirrormaker) the correct response is not to resend the message at all; if there's already a message at that offset, it's because another instance of the

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634169#comment-14634169 ] Jason Gustafson commented on KAFKA-2350: There will probably be a large number of

[jira] [Commented] (KAFKA-2275) Add a ListTopics() API to the new consumer

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634151#comment-14634151 ] Jason Gustafson commented on KAFKA-2275: [~singhashish], thanks for the response.

Re: Review Request 36570: Patch for KAFKA-2337

2015-07-20 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36570/#review92327 --- Ship it! Ship It! - Grant Henke On July 20, 2015, 9:48 p.m., Gra

[jira] [Commented] (KAFKA-2337) Verify that metric names will not collide when creating new topics

2015-07-20 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634136#comment-14634136 ] Grant Henke commented on KAFKA-2337: Updated reviewboard https://reviews.apache.org/r/

[jira] [Updated] (KAFKA-2337) Verify that metric names will not collide when creating new topics

2015-07-20 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2337: --- Attachment: KAFKA-2337_2015-07-20_16:48:25.patch > Verify that metric names will not collide when crea

Re: Review Request 36570: Patch for KAFKA-2337

2015-07-20 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36570/ --- (Updated July 20, 2015, 9:48 p.m.) Review request for kafka. Bugs: KAFKA-2337

Re: Kafka Unit Test Failures on a Mac

2015-07-20 Thread Grant Henke
Thanks Ismael! I agree clear failures or no failures is optimal. I did some hacky analysis of the open files by running the tests and utilizing the lsof command. In one run of the core tests I found the following: - 4584 regular files (REG) - 376 .jar files - Not much one can/shou

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634126#comment-14634126 ] Gwen Shapira commented on KAFKA-2350: - oh, for some reason I expected heartbeats to be

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634124#comment-14634124 ] Jason Gustafson commented on KAFKA-2350: Sure, I just meant that if you fail to ca

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

2015-07-20 Thread Flavio Junqueira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634117#comment-14634117 ] Flavio Junqueira commented on KAFKA-2260: - I like the use of an array to increase

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634114#comment-14634114 ] Gwen Shapira commented on KAFKA-2350: - Cool feature :) Can you clarify: " If you skip

Kafka High level consumer rebalancing

2015-07-20 Thread Pranay Agarwal
Hi all, Is there any way I can force Zookeeper/Kafka to rebalance new consumers only for subset of total number of partitions. I have a situation where out of 120 partitions 60 have been already consumed, but the zookeeper also assigns these empty/inactive partitions as well for the re-balancing,

[jira] [Updated] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2350: --- Description: There are some use cases in stream processing where it is helpful to be able to

[jira] [Created] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-20 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-2350: -- Summary: Add KafkaConsumer pause capability Key: KAFKA-2350 URL: https://issues.apache.org/jira/browse/KAFKA-2350 Project: Kafka Issue Type: Improvement

Re: Review Request 36570: Patch for KAFKA-2337

2015-07-20 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36570/#review92323 --- Ship it! LGTM. Just a small nit. core/src/main/scala/kafka/admin

[jira] [Commented] (KAFKA-2275) Add a ListTopics() API to the new consumer

2015-07-20 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634080#comment-14634080 ] Ashish K Singh commented on KAFKA-2275: --- I guess, it will be better to keep the desi

[jira] [Commented] (KAFKA-2299) kafka-patch-review tool does not correctly capture testing done

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634068#comment-14634068 ] Gwen Shapira commented on KAFKA-2299: - [~nehanarkhede] - mind if I review? > kafka-pa

[jira] [Commented] (KAFKA-313) Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-07-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634069#comment-14634069 ] Gwen Shapira commented on KAFKA-313: [~nehanarkhede] - mind if I review? > Add JSON/CS

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Ashish Singh
> On July 19, 2015, 1:11 a.m., Jason Gustafson wrote: > > > > Ashish Singh wrote: > Jason, thanks for your review! I looked into ConsumerNetworkClient/ > NetwrokClient, Metadata and Cluster classes. On receiving metadataUpdate, > cluster instance in metadata is updated. However, when a top

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Flavio P JUNQUEIRA
I'm with you on the races that could happen in the scenarios you describe, but I'm still not convinced that conditionally updating is the best call. Instead of conditionally updating, the broker could fence off the old owner to avoid spurious writes, and that's valid for all attempts. The advantage

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Flavio P JUNQUEIRA
I'm with you on the races that could happen in the scenarios you describe, but I'm still not convinced that conditionally updating is the best call. Instead of conditionally updating, the broker could fence off the old owner to avoid spurious writes, and that's valid for all attempts. The advantage

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Jay Kreps
It would be worth fleshing out the use cases a bit more and thinking through the overlap with the other proposals for transactions and idempotence (since likely we will end up with both). The advantage of this proposal is that it is really simple. If we go through use cases: 1. Stream processing:

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92316 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 19

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Flavio P JUNQUEIRA
Up to Ben to clarify, but I'd think that in this case, it is up to the logic of B to decide what to do. B knows that the offset isn't what it expects, so it can react accordingly. If it chooses to try again, then it should not violate any application invariant. -Flavio On Fri, Jul 17, 2015 at 9:4

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92315 --- clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsum

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92313 --- clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsum

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92309 --- clients/src/main/java/org/apache/kafka/clients/NetworkClient.java (

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92307 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 18

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Jason Gustafson
> On July 19, 2015, 1:11 a.m., Jason Gustafson wrote: > > > > Ashish Singh wrote: > Jason, thanks for your review! I looked into ConsumerNetworkClient/ > NetwrokClient, Metadata and Cluster classes. On receiving metadataUpdate, > cluster instance in metadata is updated. However, when a top

[GitHub] kafka pull request: Added changes so that bin/*.sh files can work ...

2015-07-20 Thread aloklal99
Github user aloklal99 closed the pull request at: https://github.com/apache/kafka/pull/13 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[jira] [Resolved] (KAFKA-1230) shell script files under bin don't work with cygwin (bash on windows)

2015-07-20 Thread Alok Lal (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alok Lal resolved KAFKA-1230. - Resolution: Cannot Reproduce > shell script files under bin don't work with cygwin (bash on windows) > ---

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Ben Kirwin
Hi Jun, Thanks for the close reading! Responses inline. > Thanks for the write-up. The single producer use case you mentioned makes > sense. It would be useful to include that in the KIP wiki. Great -- I'll make sure that the wiki is clear about this. > 1. What happens when the leader of the pa

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92304 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 19

[jira] [Updated] (KAFKA-1690) new java producer needs ssl support as a client

2015-07-20 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1690: -- Attachment: KAFKA-1690_2015-07-20_11:59:57.patch > new java producer needs ssl s

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-07-20 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633941#comment-14633941 ] Sriharsha Chintalapani commented on KAFKA-1690: --- Updated reviewboard https:/

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-20 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/ --- (Updated July 20, 2015, 7 p.m.) Review request for kafka. Bugs: KAFKA-1690

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-20 Thread Jun Rao
The per-key generalization is useful. As Jiangjie mentioned in KAFKA-2260, one thing that we need to sort out is what happens if a produce request has messages with different keys and some of the messages have expected offsets while some others don't. Currently, the produce response has an error co

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92297 --- clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsum

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92302 --- clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.ja

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92301 --- clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsum

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92298 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 19

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92299 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 19

Re: Review Request 36570: Patch for KAFKA-2337

2015-07-20 Thread Ashish Singh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36570/#review92296 --- Ship it! Ship It! - Ashish Singh On July 20, 2015, 5:37 p.m., Gr

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92295 --- clients/src/main/java/org/apache/kafka/clients/consumer/MockConsume

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92291 --- clients/src/main/java/org/apache/kafka/common/Cluster.java (line 18

Re: Review Request 36590: Patch for KAFKA-2275

2015-07-20 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36590/#review92294 --- clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsum

  1   2   >