Re: Review Request 34241: Patch for KAFKA-2190

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34241/#review84276 --- Ship it! core/src/main/scala/kafka/tools/MirrorMaker.scala

Re: Review Request 24214: Patch for KAFKA-1374

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24214/#review84278 --- Thanks for the updated patch. This looks good. I ended up rebasing w

Re: Review Request 33065: Patch for KAFKA-1928

> On May 18, 2015, 4:53 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Selector.java, line > > 115 > > > > > > Do we need to turn this into a map? NetworkReceive already has the > > c

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1374: -- Attachment: KAFKA-1374.patch > LogCleaner (compaction) does not support compressed topics >

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549998#comment-14549998 ] Joel Koshy commented on KAFKA-1374: --- Created reviewboard https://reviews.apache.org/r/34

Review Request 34397: Patch for KAFKA-1374

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/ --- Review request for kafka. Bugs: KAFKA-1374 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550019#comment-14550019 ] Joel Koshy commented on KAFKA-1778: --- Jun - I think this is more for convenience/debuggin

Re: Review Request 33065: Patch for KAFKA-1928

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 19, 2015, 8:26 a.m.) Review request for kafka. Bugs: 1928 and KA

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-19_11:26:18.patch > Move kafka.network over to using the network clas

[jira] [Commented] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550030#comment-14550030 ] Gwen Shapira commented on KAFKA-1928: - Updated reviewboard https://reviews.apache.org/

Review Request 34403: Patch for KAFKA-2198

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- Review request for kafka. Bugs: KAFKA-2198 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Assignee: Manikumar Reddy Status: Patch Available (was: Open) > kafka-topics.sh exits w

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198.patch > kafka-topics.sh exits with 0 status on failures > -

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550100#comment-14550100 ] Manikumar Reddy commented on KAFKA-2198: Created reviewboard https://reviews.apach

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550120#comment-14550120 ] Manikumar Reddy commented on KAFKA-2198: Case 1: Zookeeper chroot path is optional

[jira] [Created] (KAFKA-2201) Open file handle leak

Albert Visagie created KAFKA-2201: - Summary: Open file handle leak Key: KAFKA-2201 URL: https://issues.apache.org/jira/browse/KAFKA-2201 Project: Kafka Issue Type: Bug Components: c

[jira] [Commented] (KAFKA-2134) Producer blocked on metric publish

[ https://issues.apache.org/jira/browse/KAFKA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550190#comment-14550190 ] Manikumar Reddy commented on KAFKA-2134: metricChange() method will be called duri

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550352#comment-14550352 ] Bob Halley commented on KAFKA-2198: --- I realize that the chroot path is optional, but it'

Re: Review Request 34403: Patch for KAFKA-2198

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- (Updated May 19, 2015, 12:59 p.m.) Review request for kafka. Bugs: KAFKA-2198

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550387#comment-14550387 ] Manikumar Reddy commented on KAFKA-2198: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198_2015-05-19_18:27:01.patch > kafka-topics.sh exits with 0 status on fail

Re: Review Request 34403: Patch for KAFKA-2198

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- (Updated May 19, 2015, 1:14 p.m.) Review request for kafka. Bugs: KAFKA-2198

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550407#comment-14550407 ] Manikumar Reddy commented on KAFKA-2198: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198_2015-05-19_18:41:25.patch > kafka-topics.sh exits with 0 status on fail

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550412#comment-14550412 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550411#comment-14550411 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550413#comment-14550413 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk (KAF

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). ) > kafka-topic

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). ) > kafka-topic

[jira] [Created] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

Micael Capitão created KAFKA-2202: - Summary: ConsumerPerformance reports a throughput much higher than the actual one Key: KAFKA-2202 URL: https://issues.apache.org/jira/browse/KAFKA-2202 Project: Kaf

[jira] [Updated] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

[ https://issues.apache.org/jira/browse/KAFKA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micael Capitão updated KAFKA-2202: -- Description: I've been using the kafka.tools.ConsumerPerformance tool for some benchmarking unt

[jira] [Updated] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Huus updated KAFKA-2147: - Attachment: purgatory.log.gz Results from testing the most recent branch are *very* promising. We will rol

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2198: --- Reviewer: Neha Narkhede > kafka-topics.sh exits with 0 status on failures > --

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1928: --- Reporter: Jun Rao (was: Jay Kreps) > Move kafka.network over to using the network classes in > org.apache.ka

Review Request 34415: Patch for KAFKA-2195

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34415/ --- Review request for kafka. Bugs: KAFKA-2195 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Attachment: KAFKA-2195.patch > Add versionId to AbstractRequest.getErrorResponse and > Abst

[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550682#comment-14550682 ] Andrii Biletskyi commented on KAFKA-2195: - Created reviewboard https://reviews.apa

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Status: Patch Available (was: Open) > Add versionId to AbstractRequest.getErrorResponse and

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2199: --- Reviewer: Jun Rao > Make signing artifacts optional, setting maven repository possible from > command line >

Re: [Discussion] Using Client Requests and Responses in Server

Guys, I've just uploaded the patch which aligns versionId in getErrorResponse. (https://issues.apache.org/jira/browse/KAFKA-2195) Would be great if someone could review it because it's a blocker for other requests including MetadataRequest which I have to update to V1 for KIP-4. Thanks, Andrii Bi

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1928: --- Reviewer: Jun Rao (was: Jay Kreps) > Move kafka.network over to using the network classes in > org.apache.ka

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Reviewer: Guozhang Wang (was: Joel Koshy) > LogCleaner (compaction) does not support compressed t

[jira] [Updated] (KAFKA-1906) Default the Kafka log directory to a relative path within the Kafka installation directory

[ https://issues.apache.org/jira/browse/KAFKA-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1906: --- Reviewer: Jay Kreps > Default the Kafka log directory to a relative path within the Kafka > installation dire

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2196: --- Reviewer: Joel Koshy > remove roundrobin identical topic constraint in consumer coordinator >

[jira] [Updated] (KAFKA-2190) Incorporate close(timeout) to Mirror Maker

[ https://issues.apache.org/jira/browse/KAFKA-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2190: --- Reviewer: Guozhang Wang > Incorporate close(timeout) to Mirror Maker > ---

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2191: --- Reviewer: Guozhang Wang > Measured rate should not be infinite > > >

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2189: --- Reviewer: Jun Rao > Snappy compression of message batches less efficient in 0.8.2.1 >

[jira] [Updated] (KAFKA-976) Order-Preserving Mirror Maker Testcase

[ https://issues.apache.org/jira/browse/KAFKA-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-976: -- Status: Open (was: Patch Available) The current patch is too old. > Order-Preserving Mirror Maker Testcase > --

[jira] [Updated] (KAFKA-2188) JBOD Support

[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2188: --- Reviewer: Jay Kreps > JBOD Support > > > Key: KAFKA-2188 > URL: h

[jira] [Updated] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1907: --- Reviewer: Jun Rao > ZkClient can block controlled shutdown indefinitely >

Re: jira backlogs

Thanks, Joel. Do you know why it doesn't seem to reflect the recent changes? For example, kafka-1928 still has jkreps listed as the reviewer. Jun On Mon, May 18, 2015 at 3:10 PM, Joel Koshy wrote: > Here is (roughly) what I had in mind for this: > > https://cwiki.apache.org/confluence/display/K

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2195: --- Reviewer: Jun Rao > Add versionId to AbstractRequest.getErrorResponse and > AbstractRequest.getRequest >

[jira] [Commented] (KAFKA-2201) Open file handle leak

[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550716#comment-14550716 ] Manikumar Reddy commented on KAFKA-2201: On Linux machines, tuning system ulimits

[jira] [Comment Edited] (KAFKA-2201) Open file handle leak

[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550716#comment-14550716 ] Manikumar Reddy edited comment on KAFKA-2201 at 5/19/15 4:29 PM: ---

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

> On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 > > > > > > I feel this logic's irregular assignment result may make it a bit > > harder

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550781#comment-14550781 ] Abhishek Nigam commented on KAFKA-1778: --- Jun, The way I see it pinning the controlle

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-2191: - Attachment: KAFKA-2191.patch > Measured rate should not be infinite >

Review Request 34418: Patch for KAFKA-2191

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/ --- Review request for kafka. Bugs: KAFKA-2191 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550787#comment-14550787 ] Jay Kreps commented on KAFKA-2191: -- Created reviewboard https://reviews.apache.org/r/3441

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550791#comment-14550791 ] Jay Kreps commented on KAFKA-2191: -- Hey [~lindong] here is a patch that shows what I'm th

Re: Review Request 34170: Patch for KAFKA-2191

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341 --- clients/src/main/java/org/apache/kafka/common/metrics/stats/Sampled

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

> On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 > > > > > > I feel this logic's irregular assignment result may make it a bit > > harder

Re: jira backlogs

Looks like a bug - maybe browser caching. The previous jira/confluence integration allowed you to refresh the table on the page directly. I will ping infra. For now I just changed it to use the link view, which just takes you to the jira. On Tue, May 19, 2015 at 9:23 AM, Jun Rao wrote: > Thanks,

Re: Review Request 34170: Patch for KAFKA-2191

> On May 19, 2015, 5:14 p.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, > > line 46 > > > > > > Not sure if I grok why we need this while loop. Take a look at th

Re: Review Request 34170: Patch for KAFKA-2191

> On May 19, 2015, 5:14 p.m., Jay Kreps wrote: > > Oh sure. Let me look through your patch. Thank you. - Dong --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341 -

Re: jira backlogs

Just got a chance to take a look at this, so it takes me directly to JIRA. However, I think I know what you had in mind for this, Joel and it seems very useful :-) On Tue, May 19, 2015 at 10:21 AM, Joel Koshy wrote: > Looks like a bug - maybe browser caching. The previous jira/confluence > integ

Re: Review Request 34418: Patch for KAFKA-2191

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/#review84344 --- Looks good. minor comment clients/src/main/java/org/apache/kafka/c

Re: Review Request 34418: Patch for KAFKA-2191

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/#review84349 --- clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.ja

Re: [VOTE] KIP-21 Dynamic Configuration

Hey Aditya, Two comments: 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does make sense to allow setting config during topic creation. I agree with your summary that having alter topic and alter config may be confusing, but there are also some non-config changes such as rep

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

I think this looks good. What I think is missing is an overview of the timeouts from the user's perspective. My worry is that it is quite complicated to reason about the current set of timeouts. Currently we have timeout.ms metadata.fetch.timeout.ms The proposed settings I think are: batc

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

So what I understand is that, we would have 3 time outs : 1) replication timeout 2) request timeout 3) metadata timeout (existing) The request timeout has to be greater than the replication timeout. request timeout is for messages already sent to kafka and the producer is waiting for them. Thanks

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Here is the concern I had with reusing the metadata.fetch.timeout.ms: Previously people were using this as a bound on the time send() would block. It isn't a bound on the time we will wait on a metadata request, just the time the send call will block if metadata is missing for the topic. We told p

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

So the alternative to consider would be to instead have max.block.ms (or something) request.timeout replication.timeout I think this better captures what the user cares about. Here is how it would work. *max.send.block.ms * is the bound on the maximum time the p

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Hey Jay, I think that is a very reasonable concern. So the current behavior for those users are: 1. Send() will go through as long as metadata is available. 2. Send() will throw exception if the metadata of a partition is lost after pre-initialization. 3. The messages in accumulator will not be fa

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Hi Jiangjie, So when you say : It is probably better to fail the messages in accumulator than keeping them in that case because I really cannot think of any case where metadata of a partition can disappear and come up again shortly, This is true * unless there is a metadata refresh that occurs du

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Hey Jay, That is also a viable solution. I think the main purpose is to let user know how long they can block, which is important. I have some question over the proposal, though. Will user still need to send linger.ms? Will request timeout cover linger.ms as well? My concern of letting request t

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

IMO, having 4 different timeouts makes it confusing for the user and it requires the client to understand the internals of kafka. We should have a single timeout from the users perspective and handle other timeouts internally like a batch timeout. Mayuresh On Tue, May 19, 2015 at 12:42 PM, Jiangj

[jira] [Commented] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551102#comment-14551102 ] Olson,Andrew commented on KAFKA-2189: - We've been running with snappy-java 1.1.1.7 in

Notes from KIP Hangout, May 19, 2015

Hey Guys, Below are my notes from today’s hangout. Feel free to add/modify the content. KIP Hangout (May 19, 2015) KIP-12 (sasl/ssl authentication): status check - Gwen has posted patch for KAFKA-1928, network refactoring. - Needed for sasl changes. - Pretty close to completion. - Har

RE: [VOTE] KIP-21 Dynamic Configuration

Updating the discussion with the latest comments. 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll update KIP-21 with details on these. 2. Discussed during the KIP hangout. We are in agreement. (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP ca

Re: Writing a client: Connection pooling

Hi Warren, Now I get your point. Today the request ordering guarantee is enforced on the broker's socket server such that if once a request is read from one socket, the socket key is set to not readable until that request's response is written back to the socket. This behavior could be changed pr

[jira] [Created] (KAFKA-2203) Get gradle build to work with Java 8

Gaju Bhat created KAFKA-2203: Summary: Get gradle build to work with Java 8 Key: KAFKA-2203 URL: https://issues.apache.org/jira/browse/KAFKA-2203 Project: Kafka Issue Type: Bug Componen

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Fix Version/s: 0.8.1.2 Affects Version/s: 0.8.1.1 Status: Patch Available (was: Ope

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Attachment: 0001-Special-case-java-8-and-javadoc-handling.patch > Get gradle build to work with Java 8 > -

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551206#comment-14551206 ] Guozhang Wang commented on KAFKA-2160: -- Ping [~junrao] for another look at this patch

Re: [VOTE] KIP-21 Dynamic Configuration

Hi, Sorry I wasn't able to participate. I don't have objections about removing config changes from AlterTopic (as I understand both AddedConfig and DeletedConfig) - you are welcome to update the KIP page. Thanks, Andrii Biletskyi On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < aaurad...@link

Re: Review Request 34397: Patch for KAFKA-1374

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/#review84390 --- About duplicating the compression logic, one way would be: 1. Add t

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Status: In Progress (was: Patch Available) > LogCleaner (compaction) does not support compressed

Re: jira backlogs

It showed cached data for me until I logged in, then it refreshed on the next view. Not very intuitive, but if you know you need to be logged in it's probably just an annoyance that it might show old, cached data. On Tue, May 19, 2015 at 10:29 AM, Neha Narkhede wrote: > Just got a chance to take

[jira] [Created] (KAFKA-2204) Dynamic Configuration via ZK

Aditya Auradkar created KAFKA-2204: -- Summary: Dynamic Configuration via ZK Key: KAFKA-2204 URL: https://issues.apache.org/jira/browse/KAFKA-2204 Project: Kafka Issue Type: New Feature

[jira] [Created] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

Aditya Auradkar created KAFKA-2205: -- Summary: Generalize TopicConfigManager to handle multiple entity configs Key: KAFKA-2205 URL: https://issues.apache.org/jira/browse/KAFKA-2205 Project: Kafka

[jira] [Created] (KAFKA-2206) Add AlterConfig and DescribeConfig requests to Kafka

Aditya Auradkar created KAFKA-2206: -- Summary: Add AlterConfig and DescribeConfig requests to Kafka Key: KAFKA-2206 URL: https://issues.apache.org/jira/browse/KAFKA-2206 Project: Kafka Issue

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

Hey Mayuresh, I think our purpose is to find out the simplest user interface but still support all the flexibilities user might need. In that sense, we have to expose 1. linger.ms for batching purpose, 2. request timeout to support different RTT. 3. Some blocking timeout for send() Etc,.. So havi

Re: Review Request 33065: Patch for KAFKA-1928

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/#review84398 --- Thanks for the latest patch. Only a few more minor comments. Also,

Re: Review Request 33065: Patch for KAFKA-1928

> On May 18, 2015, 4:53 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Send.java, line 56 > > > > > > This needs to be long since NetworkSend always includes a size buffer, > > plus the

Re: Review Request 34397: Patch for KAFKA-1374

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/ --- (Updated May 20, 2015, 12:20 a.m.) Review request for kafka. Bugs: KAFKA-1374

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551535#comment-14551535 ] Joel Koshy commented on KAFKA-1374: --- Updated reviewboard https://reviews.apache.org/r/34

Re: Review Request 34397: Patch for KAFKA-1374

> On May 19, 2015, 9:41 p.m., Guozhang Wang wrote: > > About duplicating the compression logic, one way would be: > > > > 1. Add the following functions in ByteBufferMessageSet as: > > > > fill(buffer: ByteBuffer, compressionCodec: CompressionCodec, messages: > > MessageAndOffset*): Int > > >

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1374: -- Attachment: KAFKA-1374_2015-05-19_17:20:44.patch > LogCleaner (compaction) does not support compressed t

Review Request 34450: Fix KAFKA-2160

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34450/ --- Review request for kafka. Bugs: KAFKA-2017 https://issues.apache.org/jira/b

  1   2   >