Re: Review Request 33088: add heartbeat to coordinator

2015-04-24 Thread Onur Karaman
> On April 22, 2015, 2:33 a.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala, line 32 > > > > > > Move kafka imports above scala / external lib imports. Sorry I made the rb a

Re: Review Request 33088: add heartbeat to coordinator

2015-04-24 Thread Onur Karaman
> On April 22, 2015, 2:33 a.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala, line 123 > > > > > > How about handleConsumerJoinGroup? I agree with one of Jay's earlier comme

[jira] [Commented] (KAFKA-1334) Add failure detection capability to the coordinator / consumer

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512277#comment-14512277 ] Onur Karaman commented on KAFKA-1334: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 33088: add heartbeat to coordinator

2015-04-24 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33088/ --- (Updated April 25, 2015, 5:46 a.m.) Review request for kafka. Bugs: KAFKA-133

[jira] [Updated] (KAFKA-1334) Add failure detection capability to the coordinator / consumer

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1334: Attachment: KAFKA-1334_2015-04-24_22:46:15.patch > Add failure detection capability to the coordinat

Re: Review Request 33125: Add comment to timing fix

2015-04-24 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33125/#review81581 --- clients/src/test/java/org/apache/kafka/clients/MetadataTest.java

[jira] [Commented] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512134#comment-14512134 ] Onur Karaman commented on KAFKA-1809: - As a follow-up: there was inconsistent document

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Joel Koshy
To turn it off/on we can just add a clear config (quota.enforcement.enabled) or similar. Thanks, Joel On Fri, Apr 24, 2015 at 06:27:22PM -0400, Gari Singh wrote: > If we can't disable it, then can we use the tried and true method of using > "-1" to indicate that no throttling should take place?

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Joel Koshy
I think Jay meant a catch-all request/sec limit on all requests per-client. That makes sense. On Fri, Apr 24, 2015 at 11:02:29PM +, Aditya Auradkar wrote: > I think Joel's suggestion is quite good. It's still possible to throttle > other types of requests using purgatory but we will need a se

[jira] [Commented] (KAFKA-2132) Move Log4J appender to clients module

2015-04-24 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512040#comment-14512040 ] Ashish K Singh commented on KAFKA-2132: --- Btw, looking more into it, I guess we are c

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Aditya Auradkar
I think Joel's suggestion is quite good. It's still possible to throttle other types of requests using purgatory but we will need a separate purgatory and DelayedOperation variants of different request types or perhaps add a ThrottledOperation type. It also addresses a couple of special case sit

[jira] [Updated] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-2149: Resolution: Not A Problem Status: Resolved (was: Patch Available) > fix default InterBroker

[jira] [Commented] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511943#comment-14511943 ] Onur Karaman commented on KAFKA-2149: - Cool thanks! > fix default InterBrokerProtocol

[jira] [Commented] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511937#comment-14511937 ] Gwen Shapira commented on KAFKA-2149: - Oops, the kip actually contained both options a

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Gari Singh
If we can't disable it, then can we use the tried and true method of using "-1" to indicate that no throttling should take place? On Tue, Apr 21, 2015 at 4:38 PM, Joel Koshy wrote: > In either approach I'm not sure we considered being able to turn it > off completely. IOW, no it is not a "plugin

Re: Review Request 27204: Patch for KAFKA-1683

2015-04-24 Thread Sriharsha Chintalapani
> On April 24, 2015, 7:07 p.m., Gari Singh wrote: > > 1) I think that Session should take a Subject rather than just a single > > Principal. The reason for this is because a Subject can have multiple > > Principals (for example both a username and a group or perhaps someone > > would want to

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Jay Kreps
Hey Jun/Joel, Yeah we will definitely want to quota non-produce/consume requests. Especially offset commit and any other requests the consumer can trigger could easily get invoked in a tight loop by accident. We haven't talked about this a ton, but presumably the mechanism for all these would just

[jira] [Commented] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511899#comment-14511899 ] Onur Karaman commented on KAFKA-2149: - Okay that sounds fair. I hadn't seen the upgrad

[jira] [Commented] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511790#comment-14511790 ] Gwen Shapira commented on KAFKA-2149: - Actually the decision to default to latest vers

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-24 Thread Jun Rao
Joel, What you suggested makes sense. Not sure if there is a strong need to throttle TMR though since it should be infrequent. Thanks, Jun On Tue, Apr 21, 2015 at 12:21 PM, Joel Koshy wrote: > Given the caveats, it may be worth doing further investigation on the > alternate approach which is

Build failed in Jenkins: Kafka-trunk #472

2015-04-24 Thread Apache Jenkins Server
See Changes: [jjkoshy] KAFKA-2138; Fix producer to honor retry backoff; reviewed by Joel Koshy and Guozhang Wang -- [...truncated 2156 lines...] kafka.log.OffsetIndexTest > lookupExtremeCases PASSED

Re: Review Request 33532: fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33532/ --- (Updated April 24, 2015, 8:23 p.m.) Review request for kafka. Bugs: KAFKA-214

[jira] [Commented] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511660#comment-14511660 ] Onur Karaman commented on KAFKA-1809: - I hit problems upgrading to this patch because

[jira] [Commented] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511657#comment-14511657 ] Onur Karaman commented on KAFKA-2149: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-2149: Attachment: KAFKA-2149.patch > fix default InterBrokerProtocolVersion >

[jira] [Updated] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-2149: Status: Patch Available (was: Open) > fix default InterBrokerProtocolVersion >

[jira] [Created] (KAFKA-2149) fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman (JIRA)
Onur Karaman created KAFKA-2149: --- Summary: fix default InterBrokerProtocolVersion Key: KAFKA-2149 URL: https://issues.apache.org/jira/browse/KAFKA-2149 Project: Kafka Issue Type: Bug

Review Request 33532: fix default InterBrokerProtocolVersion

2015-04-24 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33532/ --- Review request for kafka. Bugs: KAFKA-2149 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-2138) KafkaProducer does not honor the retry backoff time.

2015-04-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2138: -- Resolution: Fixed Status: Resolved (was: Patch Available) Discussed offline with [~becket_qin]

Re: Review Request 27204: Patch for KAFKA-1683

2015-04-24 Thread Gari Singh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27204/#review81522 --- 1) I think that Session should take a Subject rather than just a sin

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
I will move the comments about subject versus principal wrt session to the PR above. The comments around keys, etc are more appropriate there. If I tie this together with my comments in the thread on SASL / Kerberos, what I am having a hard time figuring out are the pluggable framework for both a

[jira] [Commented] (KAFKA-2132) Move Log4J appender to clients module

2015-04-24 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511517#comment-14511517 ] Ashish K Singh commented on KAFKA-2132: --- [~jkreps], [~charmalloc] thanks for the inp

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Sorry Gwen, completely misunderstood the question :-). * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? Yes in current proposal. I did not see an API to create group but if you have a READ permission on a TOPIC and WRITE

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sorry, for the confusion. I'm not sure my last email is clear enough either... Consumers will have a Principal which may belong to a group. But consumer configuration also have a group.id, which controls how partitions are shared between consumers and how offsets are committed. I'm talking about t

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
We are not talking about same Groups :) I meant, Groups of consumers (which KIP-11 lists as a separate resource in the Privilege table) On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt wrote: > I see Groups as something we can add incrementally in the current model. > The acls take principalTy

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
I see Groups as something we can add incrementally in the current model. The acls take principalType: name so groups can be represented as group: groupName. We are not managing group memberships anywhere in kafka and I don’t see the need to do so. So for a topic1 using the CLI an admin can add an

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for your comments Gari. My responses are inline. Thanks Parth On 4/24/15, 10:36 AM, "Gari Singh" wrote: >Sorry - fat fingered send ... > > >Not sure if my "newbie" vote will count, but I think you are getting >pretty >close here. > >Couple of things: > >1) I know the Session object is fr

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Thanks. One more thing I'm missing in the KIP is details on the Group resource (I think we discussed this and it was just not fully updated): * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? * Will the CLI tool be used to manag

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Sorry - fat fingered send ... Not sure if my "newbie" vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this

[jira] [Commented] (KAFKA-1936) Track offset commit requests separately from produce requests

2015-04-24 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511392#comment-14511392 ] Dong Lin commented on KAFKA-1936: - Thanks! I will work on this ticket. On Fri, Apr 24, 20

[jira] [Updated] (KAFKA-1940) Initial checkout and build failing

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-1940: --- Attachment: zinc-upgrade.patch Reattach the patch file without colored diff. > Initial checko

[jira] [Updated] (KAFKA-1940) Initial checkout and build failing

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-1940: --- Attachment: (was: zinc-upgrade.patch) > Initial checkout and build failing > -

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Not sure if my "newbie" vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this is because a Subject can have m

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Sriharsha Chintalapani
+1 (non-binding) --  Harsha On April 24, 2015 at 9:59:09 AM, Parth Brahmbhatt (pbrahmbh...@hortonworks.com) wrote: You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, "Gwen Shapira" wrote: >Sample ACL JSON and Zookeeper is in public API,

[jira] [Updated] (KAFKA-2105) NullPointerException in client on MetadataRequest

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-2105: --- Attachment: guard-from-null.patch Sorry, First time I did patch with --color option. As for me

[jira] [Updated] (KAFKA-2105) NullPointerException in client on MetadataRequest

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-2105: --- Attachment: (was: guard-from-null.patch) > NullPointerException in client on MetadataReque

Re: Should 0.8.3 consumers correctly function with 0.8.2 brokers?

2015-04-24 Thread Neha Narkhede
Yes, I was clearly confused :-) On Fri, Apr 24, 2015 at 9:37 AM, Sean Lydon wrote: > Thanks for the responses. Ewen is correct that I am referring to the > *new* consumer (org.apache.kafka.clients.consumer.KafkaConsumer). > > I am extending the consumer to allow my applications more control over

[jira] [Updated] (KAFKA-1936) Track offset commit requests separately from produce requests

2015-04-24 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-1936: --- Assignee: Dong Lin (was: Aditya Auradkar) > Track offset commit requests separately from prod

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, "Gwen Shapira" wrote: >Sample ACL JSON and Zookeeper is in public API, but I thought it is >part of DefaultAuthorizer (Since Sentry and Argus won't be using >Zookeeper). >Am I wrong? Or is it the KI

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sample ACL JSON and Zookeeper is in public API, but I thought it is part of DefaultAuthorizer (Since Sentry and Argus won't be using Zookeeper). Am I wrong? Or is it the KIP? On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt wrote: > Thanks for clarifying Gwen, KIP updated. > > I tried to make th

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for clarifying Gwen, KIP updated. I tried to make the distinction by creating a section for all public APIs https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+In terface#KIP-11-AuthorizationInterface-PublicInterfacesandclasses Let me know if you think there is a bette

[jira] [Commented] (KAFKA-2105) NullPointerException in client on MetadataRequest

2015-04-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511317#comment-14511317 ] Gwen Shapira commented on KAFKA-2105: - I'm having trouble applying this patch format.

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
I yet to update the KIP with my latest proposal. So give me few days to update it.  I am looking at supporting KERBEROS for the first release and going to use JAAS Login Modules to provide authentication. And will we provide a default SASL PLAIN mechanism on the server side  Yes . I’ll update the

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
+1 (non-binding) Two nitpicks for the wiki: * Heartbeat is probably a READ and not CLUSTER operation. I'm pretty sure new consumers need it to be part of a consumer group. * Can you clearly separate which parts are the API (common to every Authorizer) and which parts are DefaultAuthorizer implemen

Re: Should 0.8.3 consumers correctly function with 0.8.2 brokers?

2015-04-24 Thread Sean Lydon
Thanks for the responses. Ewen is correct that I am referring to the *new* consumer (org.apache.kafka.clients.consumer.KafkaConsumer). I am extending the consumer to allow my applications more control over committed offsets. I really want to get away from zookeeper (so using the offset storage),

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Gari Singh
Great. Sounds good. I'll re-read the KIP ASAP. Is their another KIP around authentication providers or is that being tracked here as well. For example, the SASL PLAIN mechanism carries a username and password but currently I don't know where that would be authenticated? I notice that AuthUtils

[jira] [Commented] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Reece Markowsky (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511308#comment-14511308 ] Reece Markowsky commented on KAFKA-2148: now i see it. its kafka/clients/src/mai

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Hi, I would like to open KIP-11 for voting. Thanks Parth On 4/22/15, 1:56 PM, "Parth Brahmbhatt" wrote: >Hi Jeff, > >Thanks a lot for the review. I think you have a valid point about acls >being duplicated and the simplest solution would be to modify acls class >so they hold a set of principal

[jira] [Commented] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511304#comment-14511304 ] Jay Kreps commented on KAFKA-2148: -- Basically kafka.javaapi.producer.Producer still exist

[jira] [Comment Edited] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Reece Markowsky (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511277#comment-14511277 ] Reece Markowsky edited comment on KAFKA-2148 at 4/24/15 4:13 PM: ---

[jira] [Commented] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Reece Markowsky (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511277#comment-14511277 ] Reece Markowsky commented on KAFKA-2148: Thanks Jay! > version 0.8.2 breaks seman

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
Hi Gari,        I apologize for not clear in KIP and starting discussion thread earlier.  My initial proposal( currently in KIP ) to provide PLAINTEXT, SSL and KERBEROS as individual protocol implementation.  As you mentioned at the end “In terms of message level integrity and confidentiality (not

[jira] [Resolved] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-2148. -- Resolution: Not A Problem Hey Reece, this is not the same client but rather a new client. It is not int

[jira] [Created] (KAFKA-2148) version 0.8.2 breaks semantic versioning

2015-04-24 Thread Reece Markowsky (JIRA)
Reece Markowsky created KAFKA-2148: -- Summary: version 0.8.2 breaks semantic versioning Key: KAFKA-2148 URL: https://issues.apache.org/jira/browse/KAFKA-2148 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Gari Singh
Sorry for jumping in late, but I have been trying to follow this chain as well as the updates to the KIP. I don't mean to seem critical and I may be misunderstanding the proposed implementation, but there seems to be some confusion around terminology (at least from my perspective) and I am not sur

[jira] [Created] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-04-24 Thread Evan Huus (JIRA)
Evan Huus created KAFKA-2147: Summary: Unbalanced replication can cause extreme purgatory growth Key: KAFKA-2147 URL: https://issues.apache.org/jira/browse/KAFKA-2147 Project: Kafka Issue Type: B

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Thank you, Harsha. Yes, that makes sense. Shall take a look when the KIP is finalized. Rajini On Fri, Apr 24, 2015 at 2:34 PM, Sriharsha Chintalapani wrote: > Rajini, > I am exploring this part right now. To support PLAINTEXT and SSL > as protocols and Kerberos auth as authentication o

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
Rajini,         I am exploring this part right now. To support PLAINTEXT and SSL as protocols and Kerberos auth as authentication on top of plaintext or ssl (if users want to do encryption over an auth mechanism). This is mainly influenced by SASL or GSS-API performance issue when I enable encry

Re: Metrics package discussion

2015-04-24 Thread Jun Rao
Otis, The jira for moving the broker to the new metrics is KAFKA-1930. We didn't try to do the conversion in 0.8.2 because (1) the new metrics are missing reporters for popular systems like Graphite and Ganglia; (2) the histogram support in the new metrics is a bit different and we were not sure

[jira] [Commented] (KAFKA-2146) adding partition did not find the correct startIndex

2015-04-24 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510903#comment-14510903 ] chenshangan commented on KAFKA-2146: I'd like to provide a patch soon. > adding parti

[jira] [Created] (KAFKA-2146) adding partition did not find the correct startIndex

2015-04-24 Thread chenshangan (JIRA)
chenshangan created KAFKA-2146: -- Summary: adding partition did not find the correct startIndex Key: KAFKA-2146 URL: https://issues.apache.org/jira/browse/KAFKA-2146 Project: Kafka Issue Type: B

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

2015-04-24 Thread Gianmarco De Francisci Morales
Hi, Here are the questions I think we should consider: > 1. Do we need this at all given that we have the partition argument in > ProducerRecord which gives full control? I think we do need it because this > is a way to plug in a different partitioning strategy at run time and do it > in a fairly

[jira] [Updated] (KAFKA-2105) NullPointerException in client on MetadataRequest

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-2105: --- Attachment: guard-from-null.patch The patch checks that the argument of partitionsFor is not n

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Have there been any discussions around separating out authentication and encryption protocols for Kafka endpoints to enable different combinations? In our deployment environment, we would like to use TLS for encryption, but we don't necessarily want to use certificate-based authentication of client

[jira] [Updated] (KAFKA-1940) Initial checkout and build failing

2015-04-24 Thread Daneel Yaitskov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daneel Yaitskov updated KAFKA-1940: --- Attachment: zinc-upgrade.patch I tried to upgrade zinc library up to 0.3.7 and the issue disap

[DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Harsha, Thank you for the quick response. (Sorry had missed sending this reply to the dev-list earlier).. 1. I am not sure what the new server-side code is going to look like after refactoring under KAFKA-1928. But I was assuming that there would be only one Channel implementation tha

[jira] [Updated] (KAFKA-2106) Partition balance tool between borkers

2015-04-24 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenshangan updated KAFKA-2106: --- Attachment: KAFKA-2106.patch Implement basic rehash assignment algorithm > Partition balance tool bet

[jira] [Updated] (KAFKA-2106) Partition balance tool between borkers

2015-04-24 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenshangan updated KAFKA-2106: --- Reviewer: Jun Rao Status: Patch Available (was: Open) > Partition balance tool between borkers

[jira] [Commented] (KAFKA-2106) Partition balance tool between borkers

2015-04-24 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510614#comment-14510614 ] chenshangan commented on KAFKA-2106: When adding new brokers to the cluster, existing

[jira] [Issue Comment Deleted] (KAFKA-2106) Partition balance tool between borkers

2015-04-24 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenshangan updated KAFKA-2106: --- Comment: was deleted (was: Topics in the cluster can be divided into two categories: 1. nPartitions >