Re: Review Request 34394: Patch for KAFKA-1907

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34394/ --- (Updated May 19, 2015, 5:20 a.m.) Review request for kafka. Bugs: KAFKA-1907

[jira] [Commented] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549796#comment-14549796 ] jaikiran pai commented on KAFKA-1907: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1907: Attachment: KAFKA-1907.patch > ZkClient can block controlled shutdown indefinitely > ---

[jira] [Updated] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1907: Status: Patch Available (was: In Progress) > ZkClient can block controlled shutdown indefinitely >

Review Request 34394: Patch for KAFKA-1907

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34394/ --- Review request for kafka. Bugs: KAFKA-1907 https://issues.apache.org/jira/b

Re: Writing a client: Connection pooling

cc: kafka-clients mailing list On May 18, 2015 4:24 PM, "Warren Falk" wrote: > Thanks Guozhang, > > Actually your previous email was clear and I understood it. Current broker > design means that parallel requests require parallel connections. > > But I think you misunderstood me. I am not askin

[jira] [Created] (KAFKA-2200) kafkaProducer.send() should not call callback.onCompletion()

Jiangjie Qin created KAFKA-2200: --- Summary: kafkaProducer.send() should not call callback.onCompletion() Key: KAFKA-2200 URL: https://issues.apache.org/jira/browse/KAFKA-2200 Project: Kafka Iss

Re: [DISCUSSION] Two behaviors in new producer

Hi Jay, Thanks for the explanations, Jay. That makes sense. I¹ll create a ticket to for (1). For (2), as you said it depends on traffic pattern a lot. I will try to run some benchmark to see if we want to change (2), but we might still want to make the java doc clearer if possible. Thanks, Jian

Re: [DISCUSSION] Two behaviors in new producer

Hey Becket, 1. I agree that this is a problem. I think this ended up this way because there are two ways a TimeoutException can be thrown, the server can timeout or we can timeout waiting for memory. I think the complaint at the time was that it was annoying that you needed to write the same excep

Re: Writing a client: Connection pooling

Thanks Guozhang, Actually your previous email was clear and I understood it. Current broker design means that parallel requests require parallel connections. But I think you misunderstood me. I am not asking if that is how the broker works, now, I am proposing that the broker should not work th

[DISCUSSION] Two behaviors in new producer

Hi, I have two questions when writing patch for KAFKA-2142 and needs some help: 1. In send(), we actually might call callback.onCompletion(), this might break the guarantee that callbacks are fired in sending order of a particular partition. 2. In RecordAccumulator, current logic is that

Re: jira backlogs

I forgot to add - I believe you need to be logged into jira at the time you view the page. The search basically filters by patch-available, non-empty reviewer and ordered by updated date. Thanks, Joel On Mon, May 18, 2015 at 03:10:03PM -0700, Joel Koshy wrote: > Here is (roughly) what I had in

Re: jira backlogs

Here is (roughly) what I had in mind for this: https://cwiki.apache.org/confluence/display/KAFKA/Patch+submission+and+review#Patchsubmissionandreview-Reviewqueue Earlier versions of wiki/jira integration were much better in that the inline table had pagination within the wiki page itself. Unfortun

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34273/#review84223 --- core/src/main/scala/kafka/coordinator/PartitionAssignor.scala

Re: [Discussion] Using Client Requests and Responses in Server

No worries, I think it's hard to foresee all nuances before actually implementing/writing code. I also have a feeling there will be some other issues with requests versioning so I plan to finish end-to-end MetadataRequest_V1 to ensure we don't miss anything in terms of AbstractRequest/Response API,

[jira] [Comment Edited] (KAFKA-1977) Make logEndOffset available in the Zookeeper consumer

[ https://issues.apache.org/jira/browse/KAFKA-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549252#comment-14549252 ] Vamsi Subhash Achanta edited comment on KAFKA-1977 at 5/18/15 9:21 PM: -

[jira] [Commented] (KAFKA-1977) Make logEndOffset available in the Zookeeper consumer

[ https://issues.apache.org/jira/browse/KAFKA-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549252#comment-14549252 ] Vamsi Subhash Achanta commented on KAFKA-1977: -- Hi Will, Even I have applied

Re: [Discussion] Using Client Requests and Responses in Server

Sorry for the confusion regarding errorResponses... On Mon, May 18, 2015 at 9:10 PM, Andrii Biletskyi < andrii.bilets...@stealth.ly> wrote: > Jun, > > Thanks for the explanation. I believe my understanding is close to what > you have written. I see, I still think that this approach is > somewhat

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549234#comment-14549234 ] Guozhang Wang commented on KAFKA-2160: -- Updated reviewboard https://reviews.apache.or

[jira] [Updated] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2160: - Attachment: KAFKA-2160_2015-05-18_14:07:48.patch > DelayedOperationPurgatory should remove the pai

Re: Review Request 34283: Third Attempt: use a global read-write lock in purgatory; remove join-group purgatory from coordinator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34283/ --- (Updated May 18, 2015, 9:08 p.m.) Review request for kafka. Bugs: KAFKA-2160

Re: Review Request 34283: Third Attempt: use a global read-write lock in purgatory; remove join-group purgatory from coordinator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34283/ --- (Updated May 18, 2015, 9:07 p.m.) Review request for kafka. Summary (updated)

RE: [VOTE] KIP-21 Dynamic Configuration

Yeah, that was just a typo. I've fixed it. Thanks for calling it out. In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic and DeleteTopic. The topic configs are a sub-type of the Create and Alter commands. I think it would be nice to simply have a AlterConfig command that ca

[jira] [Assigned] (KAFKA-2182) zkClient dies if there is any exception while reconnecting

[ https://issues.apache.org/jira/browse/KAFKA-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2182: --- Assignee: Parth Brahmbhatt > zkClient dies if there is any exception while reconnecti

Jenkins build is back to normal : Kafka-trunk #490

See

[jira] [Resolved] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect

[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-1082. Resolution: Duplicate Fix Version/s: 0.8.3 This is already resolved in KAFKA-2169. > zkclient dies a

[jira] [Commented] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549151#comment-14549151 ] Ewen Cheslack-Postava commented on KAFKA-2199: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2199: - Attachment: KAFKA-2199.patch > Make signing artifacts optional, setting maven repo

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2199: - Status: Patch Available (was: Open) > Make signing artifacts optional, setting ma

Review Request 34369: Patch for KAFKA-2199

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34369/ --- Review request for kafka. Bugs: KAFKA-2199 https://issues.apache.org/jira/b

[jira] [Created] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

Ewen Cheslack-Postava created KAFKA-2199: Summary: Make signing artifacts optional, setting maven repository possible from command line Key: KAFKA-2199 URL: https://issues.apache.org/jira/browse/KAFKA-2199

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549117#comment-14549117 ] Jun Rao commented on KAFKA-1778: [~anigam], could you explain a bit the use case of pinnin

[jira] [Updated] (KAFKA-2169) Upgrade to zkclient-0.5

[ https://issues.apache.org/jira/browse/KAFKA-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2169: --- Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) Thanks for t

Re: [VOTE] KIP-21 Dynamic Configuration

Agree with Jun here on the JSON format. I think your intention was likely to have actual JSON here and it was just a typo in the wiki? -Todd On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote: > Aditya, > > Another thing to consider. In KIP-4, we are adding a new RPC request to > change and retrie

Re: Review Request 33552: Patch for KAFKA-2142

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33552/#review84191 --- clients/src/main/java/org/apache/kafka/clients/producer/internals/R

Re: [VOTE] KIP-21 Dynamic Configuration

Aditya, Another thing to consider. In KIP-4, we are adding a new RPC request to change and retrieve topic configs. Do we want to add a similar RPC request to change configs per client id? If so, do we want to introduce a separate new request or have a combined new request for both topic and client

Re: [Discussion] Using Client Requests and Responses in Server

Hi, Andrii, Good point on the constructor for MetadataResponse. Perhaps in V2 of MetadataResponse, we can add a new constructor with both cluster and a topicConfig map. Currently, cluster doesn't really need topic configs and we can leave it as it is now. Thanks, Jun On Mon, May 18, 2015 at 11:

Re: [Discussion] Using Client Requests and Responses in Server

Jun, Thanks for the explanation. I believe my understanding is close to what you have written. I see, I still think that this approach is somewhat limiting (what if you add field of type int in V1 and then remove another field of type int in V2 - method overloading for V0 and V2 constructors will

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548343#comment-14548343 ] Jiangjie Qin commented on KAFKA-1778: - Hey [~joestein], I'm a little bit confused here

Re: Writing a client: Connection pooling

Hi Warren, Maybe I was a little misleading in my previous email: 1. The statement that "The server guarantees that on a single TCP connection, requests will be processed in the order" is still valid. On a single connection, all types of requests (produce/fetch/metadata) are handled in order. 2.

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548354#comment-14548354 ] Manikumar Reddy commented on KAFKA-1374: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1374: --- Attachment: KAFKA-1374_2015-05-18_22:55:48.patch > LogCleaner (compaction) does not support co

Re: Review Request 24214: Patch for KAFKA-1374

> On May 12, 2015, 2:01 p.m., Joel Koshy wrote: > > core/src/main/scala/kafka/log/LogCleaner.scala, line 409 > > > > > > I would suggest one of two options over this (i.e., instead of two > > helper methods) > > -

Re: Review Request 24214: Patch for KAFKA-1374

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24214/ --- (Updated May 18, 2015, 5:29 p.m.) Review request for kafka. Bugs: KAFKA-1374

Re: [Discussion] Using Client Requests and Responses in Server

Andri, Let me clarify a bit how things work now. You can see if this fits your need or if it can be improved. If you look at OffsetCommitRequest, our convention is the following. 1. The request object can be constructed from a set of required fields. The client typically constructs a request obje

Re: Next is 0.8.3 or is it 0.9?

The next version will be 0.8.3. On Sun, May 17, 2015 at 9:55 PM, Jaikiran Pai wrote: > Hello everyone, > > What is the next planned version of Kafka going to be? I vaguely remember > reading in some mail that it will be 0.9 and "trunk" is where it would be > released from. I was just updating my

Re: Review Request 33065: Patch for KAFKA-1928

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/#review84158 --- Thanks for the latest patch. Looks good. Just a few minor comments b

[VOTE] KIP-21 Dynamic Configuration

https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration Aditya

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-18_18:55:38.patch > Move kafka.network over to using the network clas

[jira] [Commented] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548197#comment-14548197 ] Gwen Shapira commented on KAFKA-1928: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 33065: Patch for KAFKA-1928

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 18, 2015, 3:56 p.m.) Review request for kafka. Bugs: 1928 and KA

Re: [Discussion] Using Client Requests and Responses in Server

Hi all, I started working on it and it seems we are going the wrong way. So it appears we need to distinguish constructors by versions in request/response (so we can set correct schema). Request/Response classes will look like: class SomeRequest extends AbstractRequest { SomeRequest(versionId,

[jira] [Created] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

Bob Halley created KAFKA-2198: - Summary: kafka-topics.sh exits with 0 status on failures Key: KAFKA-2198 URL: https://issues.apache.org/jira/browse/KAFKA-2198 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548121#comment-14548121 ] Gwen Shapira commented on KAFKA-1928: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-18_17:57:39.patch > Move kafka.network over to using the network clas

Re: Review Request 33065: Patch for KAFKA-1928

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 18, 2015, 2:58 p.m.) Review request for kafka. Bugs: 1928 and KA

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

Gianmarco,             I’ll send the patch soon. Thanks, Harsha On May 18, 2015 at 1:34:50 AM, Gianmarco De Francisci Morales (g...@apache.org) wrote: Hi, If everything is in order, can we proceed to implement it? Cheers, -- Gianmarco On 13 May 2015 at 03:06, Jiangjie Qin wrote:

Re: Review Request 33620: Patch for KAFKA-1690

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review84144 --- clients/src/main/java/org/apache/kafka/common/config/SecurityConfig

Re: Review Request 33065: Patch for KAFKA-1928

> On May 15, 2015, 10:12 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 354 > > > > > > I guess we will make use of this in a followup patch? Actually, I think its no longer neede

Re: Review Request 33065: Patch for KAFKA-1928

> On May 15, 2015, 10:12 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/MultiSend.java, line > > 74 > > > > > > Actually, with MultiSend, we will be sending a 4-byte size plus the > >

Re: Review Request 33065: Patch for KAFKA-1928

> On May 15, 2015, 10:12 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java, > > lines 159-162 > > > > > > We don't need this since the idle conenctions only need to

Re: [Vote] KIP-11 Authorization design for kafka security

+1 ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Fri, May 15, 2015 at 7:35 PM, Jun Rao wrote: > +1 > > Thanks, > > Jun > > On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt < > pbrahmbh...@hortonworks.com> wrote: > > > Hi, > > > > Op

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547838#comment-14547838 ] Joe Stein commented on KAFKA-1778: -- I was thinking that the broker when starting up would

[jira] [Commented] (KAFKA-1977) Make logEndOffset available in the Zookeeper consumer

[ https://issues.apache.org/jira/browse/KAFKA-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547733#comment-14547733 ] Will Funnell commented on KAFKA-1977: - [~vamsi360] I've tested this with 0.8.1.2 and 0

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

Hi, If everything is in order, can we proceed to implement it? Cheers, -- Gianmarco On 13 May 2015 at 03:06, Jiangjie Qin wrote: > Hi Harsha, > > If you open this link > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa > ls > > All the KIPs are the child page of t

[jira] [Commented] (KAFKA-2194) Produce request failure after Kafka + Zookeeper restart

[ https://issues.apache.org/jira/browse/KAFKA-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547704#comment-14547704 ] Ian Morgan commented on KAFKA-2194: --- Hi Jiangjie, thanks for the reply. I've tried both