Build failed in Jenkins: KafkaPreCommit #104

2015-05-19 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2196; Remove identical topic constraint in round-robin assignor; reviewed by Guozhang Wang -- [...truncated 769 lines...] at kafka.server.KafkaServer.getBroke

RE: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Aditya Auradkar
Thanks Andrii. I'll make the changes. I've also updated KIP-21 to include the new config requests. Take a look and vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration Aditya From: Andrii Biletskyi [andrii.bilets...@steal

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Ewen Cheslack-Postava
The fact that I understand the producer internals and am still struggling to understand the implications of the different settings, how I would set them, and how they potentially interact such that I could set invalid combinations seems like a red flag to me... Being able to say "I want produce req

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2191: - Reviewer: Joel Koshy (was: Guozhang Wang) [~jjkoshy] Could you help taking a look at this one or

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2196: - Resolution: Fixed Reviewer: Guozhang Wang (was: Joel Koshy) Status: Resolved (was:

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-19 Thread Sriharsha Chintalapani
> On May 15, 2015, 8:26 p.m., Michael Herstine wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > line 418 > > > > > > So you're transferring bytes from `appReadBuffer` to `ds

[jira] [Commented] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551746#comment-14551746 ] Ewen Cheslack-Postava commented on KAFKA-2203: -- +1 Double checked against JDK

Build failed in Jenkins: KafkaPreCommit #103

2015-05-19 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2160; remove watcher list if empty in purgatory; remove join-group purgatory in coordinator; reviewed by Jun Rao -- [...truncated 1354 lines...] kafka.consumer.TopicF

[jira] [Updated] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2160: - Resolution: Fixed Status: Resolved (was: Patch Available) > DelayedOperationPurgatory sho

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551671#comment-14551671 ] Guozhang Wang commented on KAFKA-2160: -- Thanks for the review, committed to trunk. >

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hi Jay, I updated what I think int KIP wiki. Just a short summary here. Because we need timeout for: 1. Send() 2. Batches in accumulator 3. Requests in flight. That means we need to have at least three configurations if we do not reuse configurations. I think we probably want to also separate th

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
Hi Jay, If I am understanding this correctly, we should treat the Batch Timeout separately from the Request Timeout. Request time out can be used only for inflight request (request that have been sent to kafka brokers and waiting for response) and Batch Timeout can be used to expire the batches in

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-19 Thread Sriharsha Chintalapani
> On May 15, 2015, 8:26 p.m., Michael Herstine wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 112-121 > > > > > > Sorry, but I'm a little confused here. Why would th

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
Yeah I think linger.ms remains separate, setting that is a performance optimization rather than failure handling thing. We should ideally sanity check this, though, in my proposal, since if they set linger.ms > request.timeout then that won't work. It's true that in my proposal that the actual rep

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-19 Thread Craig Allsop (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551556#comment-14551556 ] Craig Allsop commented on KAFKA-2147: - [~junrao] Forgive me if this is the wrong proce

[jira] [Comment Edited] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-19 Thread Craig Allsop (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551556#comment-14551556 ] Craig Allsop edited comment on KAFKA-2147 at 5/20/15 12:33 AM: -

Re: Review Request 34283: Third Attempt: use a global read-write lock in purgatory; remove join-group purgatory from coordinator

2015-05-19 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34283/#review84424 --- Ship it! Thanks for the patch. Looks good to me. Just a few minor c

[jira] [Commented] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551540#comment-14551540 ] Guozhang Wang commented on KAFKA-2017: -- Created reviewboard https://reviews.apache.or

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Reviewer: Jun Rao > Persist Coordinator State for Coordinator Failover > -

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Attachment: KAFKA-2017.patch > Persist Coordinator State for Coordinator Failover > --

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Assignee: Guozhang Wang (was: Onur Karaman) Status: Patch Available (was: Open) > Persist

Review Request 34450: Fix KAFKA-2160

2015-05-19 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34450/ --- Review request for kafka. Bugs: KAFKA-2017 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1374: -- Attachment: KAFKA-1374_2015-05-19_17:20:44.patch > LogCleaner (compaction) does not support compressed t

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551535#comment-14551535 ] Joel Koshy commented on KAFKA-1374: --- Updated reviewboard https://reviews.apache.org/r/34

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
> On May 19, 2015, 9:41 p.m., Guozhang Wang wrote: > > About duplicating the compression logic, one way would be: > > > > 1. Add the following functions in ByteBufferMessageSet as: > > > > fill(buffer: ByteBuffer, compressionCodec: CompressionCodec, messages: > > MessageAndOffset*): Int > > >

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/ --- (Updated May 20, 2015, 12:20 a.m.) Review request for kafka. Bugs: KAFKA-1374

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/#review84398 --- Thanks for the latest patch. Only a few more minor comments. Also,

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Jun Rao
> On May 18, 2015, 4:53 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Send.java, line 56 > > > > > > This needs to be long since NetworkSend always includes a size buffer, > > plus the

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hey Mayuresh, I think our purpose is to find out the simplest user interface but still support all the flexibilities user might need. In that sense, we have to expose 1. linger.ms for batching purpose, 2. request timeout to support different RTT. 3. Some blocking timeout for send() Etc,.. So havi

[jira] [Created] (KAFKA-2206) Add AlterConfig and DescribeConfig requests to Kafka

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2206: -- Summary: Add AlterConfig and DescribeConfig requests to Kafka Key: KAFKA-2206 URL: https://issues.apache.org/jira/browse/KAFKA-2206 Project: Kafka Issue

[jira] [Created] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2205: -- Summary: Generalize TopicConfigManager to handle multiple entity configs Key: KAFKA-2205 URL: https://issues.apache.org/jira/browse/KAFKA-2205 Project: Kafka

[jira] [Created] (KAFKA-2204) Dynamic Configuration via ZK

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2204: -- Summary: Dynamic Configuration via ZK Key: KAFKA-2204 URL: https://issues.apache.org/jira/browse/KAFKA-2204 Project: Kafka Issue Type: New Feature

Re: jira backlogs

2015-05-19 Thread Ewen Cheslack-Postava
It showed cached data for me until I logged in, then it refreshed on the next view. Not very intuitive, but if you know you need to be logged in it's probably just an annoyance that it might show old, cached data. On Tue, May 19, 2015 at 10:29 AM, Neha Narkhede wrote: > Just got a chance to take

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Status: In Progress (was: Patch Available) > LogCleaner (compaction) does not support compressed

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/#review84390 --- About duplicating the compression logic, one way would be: 1. Add t

Re: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Andrii Biletskyi
Hi, Sorry I wasn't able to participate. I don't have objections about removing config changes from AlterTopic (as I understand both AddedConfig and DeletedConfig) - you are welcome to update the KIP page. Thanks, Andrii Biletskyi On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < aaurad...@link

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551206#comment-14551206 ] Guozhang Wang commented on KAFKA-2160: -- Ping [~junrao] for another look at this patch

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Attachment: 0001-Special-case-java-8-and-javadoc-handling.patch > Get gradle build to work with Java 8 > -

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Fix Version/s: 0.8.1.2 Affects Version/s: 0.8.1.1 Status: Patch Available (was: Ope

[jira] [Created] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
Gaju Bhat created KAFKA-2203: Summary: Get gradle build to work with Java 8 Key: KAFKA-2203 URL: https://issues.apache.org/jira/browse/KAFKA-2203 Project: Kafka Issue Type: Bug Componen

Re: Writing a client: Connection pooling

2015-05-19 Thread Guozhang Wang
Hi Warren, Now I get your point. Today the request ordering guarantee is enforced on the broker's socket server such that if once a request is read from one socket, the socket key is set to not readable until that request's response is written back to the socket. This behavior could be changed pr

RE: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Aditya Auradkar
Updating the discussion with the latest comments. 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll update KIP-21 with details on these. 2. Discussed during the KIP hangout. We are in agreement. (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP ca

Notes from KIP Hangout, May 19, 2015

2015-05-19 Thread Ashish Singh
Hey Guys, Below are my notes from today’s hangout. Feel free to add/modify the content. KIP Hangout (May 19, 2015) KIP-12 (sasl/ssl authentication): status check - Gwen has posted patch for KAFKA-1928, network refactoring. - Needed for sasl changes. - Pretty close to completion. - Har

[jira] [Commented] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-19 Thread Olson,Andrew (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551102#comment-14551102 ] Olson,Andrew commented on KAFKA-2189: - We've been running with snappy-java 1.1.1.7 in

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
IMO, having 4 different timeouts makes it confusing for the user and it requires the client to understand the internals of kafka. We should have a single timeout from the users perspective and handle other timeouts internally like a batch timeout. Mayuresh On Tue, May 19, 2015 at 12:42 PM, Jiangj

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hey Jay, That is also a viable solution. I think the main purpose is to let user know how long they can block, which is important. I have some question over the proposal, though. Will user still need to send linger.ms? Will request timeout cover linger.ms as well? My concern of letting request t

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
Hi Jiangjie, So when you say : It is probably better to fail the messages in accumulator than keeping them in that case because I really cannot think of any case where metadata of a partition can disappear and come up again shortly, This is true * unless there is a metadata refresh that occurs du

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hey Jay, I think that is a very reasonable concern. So the current behavior for those users are: 1. Send() will go through as long as metadata is available. 2. Send() will throw exception if the metadata of a partition is lost after pre-initialization. 3. The messages in accumulator will not be fa

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
So the alternative to consider would be to instead have max.block.ms (or something) request.timeout replication.timeout I think this better captures what the user cares about. Here is how it would work. *max.send.block.ms * is the bound on the maximum time the p

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
Here is the concern I had with reusing the metadata.fetch.timeout.ms: Previously people were using this as a bound on the time send() would block. It isn't a bound on the time we will wait on a metadata request, just the time the send call will block if metadata is missing for the topic. We told p

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
So what I understand is that, we would have 3 time outs : 1) replication timeout 2) request timeout 3) metadata timeout (existing) The request timeout has to be greater than the replication timeout. request timeout is for messages already sent to kafka and the producer is waiting for them. Thanks

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
I think this looks good. What I think is missing is an overview of the timeouts from the user's perspective. My worry is that it is quite complicated to reason about the current set of timeouts. Currently we have timeout.ms metadata.fetch.timeout.ms The proposed settings I think are: batc

Re: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Jay Kreps
Hey Aditya, Two comments: 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does make sense to allow setting config during topic creation. I agree with your summary that having alter topic and alter config may be confusing, but there are also some non-config changes such as rep

Re: Review Request 34418: Patch for KAFKA-2191

2015-05-19 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/#review84349 --- clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.ja

Re: Review Request 34418: Patch for KAFKA-2191

2015-05-19 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/#review84344 --- Looks good. minor comment clients/src/main/java/org/apache/kafka/c

Re: jira backlogs

2015-05-19 Thread Neha Narkhede
Just got a chance to take a look at this, so it takes me directly to JIRA. However, I think I know what you had in mind for this, Joel and it seems very useful :-) On Tue, May 19, 2015 at 10:21 AM, Joel Koshy wrote: > Looks like a bug - maybe browser caching. The previous jira/confluence > integ

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Dong Lin
> On May 19, 2015, 5:14 p.m., Jay Kreps wrote: > > Oh sure. Let me look through your patch. Thank you. - Dong --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341 -

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Dong Lin
> On May 19, 2015, 5:14 p.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, > > line 46 > > > > > > Not sure if I grok why we need this while loop. Take a look at th

Re: jira backlogs

2015-05-19 Thread Joel Koshy
Looks like a bug - maybe browser caching. The previous jira/confluence integration allowed you to refresh the table on the page directly. I will ping infra. For now I just changed it to use the link view, which just takes you to the jira. On Tue, May 19, 2015 at 9:23 AM, Jun Rao wrote: > Thanks,

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Guozhang Wang
> On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 > > > > > > I feel this logic's irregular assignment result may make it a bit > > harder

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341 --- clients/src/main/java/org/apache/kafka/common/metrics/stats/Sampled

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550791#comment-14550791 ] Jay Kreps commented on KAFKA-2191: -- Hey [~lindong] here is a patch that shows what I'm th

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550787#comment-14550787 ] Jay Kreps commented on KAFKA-2191: -- Created reviewboard https://reviews.apache.org/r/3441

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-2191: - Attachment: KAFKA-2191.patch > Measured rate should not be infinite >

Review Request 34418: Patch for KAFKA-2191

2015-05-19 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/ --- Review request for kafka. Bugs: KAFKA-2191 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

2015-05-19 Thread Abhishek Nigam (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550781#comment-14550781 ] Abhishek Nigam commented on KAFKA-1778: --- Jun, The way I see it pinning the controlle

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Onur Karaman
> On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 > > > > > > I feel this logic's irregular assignment result may make it a bit > > harder

[jira] [Comment Edited] (KAFKA-2201) Open file handle leak

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550716#comment-14550716 ] Manikumar Reddy edited comment on KAFKA-2201 at 5/19/15 4:29 PM: ---

[jira] [Commented] (KAFKA-2201) Open file handle leak

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550716#comment-14550716 ] Manikumar Reddy commented on KAFKA-2201: On Linux machines, tuning system ulimits

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2195: --- Reviewer: Jun Rao > Add versionId to AbstractRequest.getErrorResponse and > AbstractRequest.getRequest >

Re: jira backlogs

2015-05-19 Thread Jun Rao
Thanks, Joel. Do you know why it doesn't seem to reflect the recent changes? For example, kafka-1928 still has jkreps listed as the reviewer. Jun On Mon, May 18, 2015 at 3:10 PM, Joel Koshy wrote: > Here is (roughly) what I had in mind for this: > > https://cwiki.apache.org/confluence/display/K

[jira] [Updated] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1907: --- Reviewer: Jun Rao > ZkClient can block controlled shutdown indefinitely >

[jira] [Updated] (KAFKA-2188) JBOD Support

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2188: --- Reviewer: Jay Kreps > JBOD Support > > > Key: KAFKA-2188 > URL: h

[jira] [Updated] (KAFKA-976) Order-Preserving Mirror Maker Testcase

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-976: -- Status: Open (was: Patch Available) The current patch is too old. > Order-Preserving Mirror Maker Testcase > --

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2189: --- Reviewer: Jun Rao > Snappy compression of message batches less efficient in 0.8.2.1 >

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2191: --- Reviewer: Guozhang Wang > Measured rate should not be infinite > > >

[jira] [Updated] (KAFKA-2190) Incorporate close(timeout) to Mirror Maker

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2190: --- Reviewer: Guozhang Wang > Incorporate close(timeout) to Mirror Maker > ---

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2196: --- Reviewer: Joel Koshy > remove roundrobin identical topic constraint in consumer coordinator >

[jira] [Updated] (KAFKA-1906) Default the Kafka log directory to a relative path within the Kafka installation directory

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1906: --- Reviewer: Jay Kreps > Default the Kafka log directory to a relative path within the Kafka > installation dire

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Reviewer: Guozhang Wang (was: Joel Koshy) > LogCleaner (compaction) does not support compressed t

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1928: --- Reviewer: Jun Rao (was: Jay Kreps) > Move kafka.network over to using the network classes in > org.apache.ka

Re: [Discussion] Using Client Requests and Responses in Server

2015-05-19 Thread Andrii Biletskyi
Guys, I've just uploaded the patch which aligns versionId in getErrorResponse. (https://issues.apache.org/jira/browse/KAFKA-2195) Would be great if someone could review it because it's a blocker for other requests including MetadataRequest which I have to update to V1 for KIP-4. Thanks, Andrii Bi

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2199: --- Reviewer: Jun Rao > Make signing artifacts optional, setting maven repository possible from > command line >

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Status: Patch Available (was: Open) > Add versionId to AbstractRequest.getErrorResponse and

[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550682#comment-14550682 ] Andrii Biletskyi commented on KAFKA-2195: - Created reviewboard https://reviews.apa

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Attachment: KAFKA-2195.patch > Add versionId to AbstractRequest.getErrorResponse and > Abst

Review Request 34415: Patch for KAFKA-2195

2015-05-19 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34415/ --- Review request for kafka. Bugs: KAFKA-2195 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1928: --- Reporter: Jun Rao (was: Jay Kreps) > Move kafka.network over to using the network classes in > org.apache.ka

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2198: --- Reviewer: Neha Narkhede > kafka-topics.sh exits with 0 status on failures > --

[jira] [Updated] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-19 Thread Evan Huus (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Huus updated KAFKA-2147: - Attachment: purgatory.log.gz Results from testing the most recent branch are *very* promising. We will rol

[jira] [Updated] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

2015-05-19 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micael Capitão updated KAFKA-2202: -- Description: I've been using the kafka.tools.ConsumerPerformance tool for some benchmarking unt

[jira] [Created] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

2015-05-19 Thread JIRA
Micael Capitão created KAFKA-2202: - Summary: ConsumerPerformance reports a throughput much higher than the actual one Key: KAFKA-2202 URL: https://issues.apache.org/jira/browse/KAFKA-2202 Project: Kaf

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). ) > kafka-topic

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). ) > kafka-topic

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550413#comment-14550413 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550411#comment-14550411 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550412#comment-14550412 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198_2015-05-19_18:41:25.patch > kafka-topics.sh exits with 0 status on fail

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550407#comment-14550407 ] Manikumar Reddy commented on KAFKA-2198: Updated reviewboard https://reviews.apach

Re: Review Request 34403: Patch for KAFKA-2198

2015-05-19 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- (Updated May 19, 2015, 1:14 p.m.) Review request for kafka. Bugs: KAFKA-2198

  1   2   >