Re: Review Request 32459: Patch for KAFKA-2044

> On March 26, 2015, 12:47 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java, > > line 53 > > > > > > Cluster may not serialize properly with those nulls in PartitionI

Re: Review Request 32459: Patch for KAFKA-2044

> On March 26, 2015, 12:47 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java, > > line 53 > > > > > > Cluster may not serialize properly with those nulls in PartitionI

[jira] [Updated] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

[ https://issues.apache.org/jira/browse/KAFKA-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2044: Attachment: KAFKA-2044_2015-03-25_19:20:16.patch > Support requests and responses from o.a.k.common

[jira] [Commented] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

[ https://issues.apache.org/jira/browse/KAFKA-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381232#comment-14381232 ] Gwen Shapira commented on KAFKA-2044: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 32459: Patch for KAFKA-2044

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32459/ --- (Updated March 26, 2015, 2:20 a.m.) Review request for kafka. Bugs: KAFKA-204

Review Request 32519: Patch for KAFKA-2050

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32519/ --- Review request for kafka. Bugs: KAFKA-2050 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381250#comment-14381250 ] Tim Brooks commented on KAFKA-2050: --- Created reviewboard https://reviews.apache.org/r/32

[jira] [Updated] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Brooks updated KAFKA-2050: -- Attachment: KAFKA-2050.patch > Avoid calling .size() on java.util.ConcurrentLinkedQueue > --

[jira] [Commented] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381251#comment-14381251 ] Tim Brooks commented on KAFKA-2050: --- Okay. I submitted it. I had some trouble. But I thi

Re: Review Request 32519: Patch for KAFKA-2050

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32519/#review77855 --- Ship it! Ship It! - Sriharsha Chintalapani On March 26, 2015, 2:

[jira] [Created] (KAFKA-2051) Add hashCode() and equals() to ProducerRecord

Justin Plock created KAFKA-2051: --- Summary: Add hashCode() and equals() to ProducerRecord Key: KAFKA-2051 URL: https://issues.apache.org/jira/browse/KAFKA-2051 Project: Kafka Issue Type: Improve

Re: MirrorMaker improvements

You can set the num.consumer.fetchers to be a larger number (e.g. 3) and there will be more fetchers created to fetch from different partitions in the same broker. Each fetcher will have there own TCP connection. WRT the congestion window, yes, if the link has a high packet drop rate then congest

Re: [VOTE] KIP-15 add a close with timeout to new producer

> > We have agreed that we will have an error log to inform user about this > mis-usage. The options differ in the way how we can force user to take a > look at that error log. Since we have to detect the problem in order to log an appropriate error message, we have a way to tell if the user is d

Contributing a KIP

Hi all, I'd like to put forth a KIP but don't seem to have write access to cwiki. Do I need to get a special permission bit set, or should I simply open up the discussion here on the mailing list? Cheers, Tom

[jira] [Commented] (KAFKA-2051) Add hashCode() and equals() to ProducerRecord

[ https://issues.apache.org/jira/browse/KAFKA-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381321#comment-14381321 ] Ewen Cheslack-Postava commented on KAFKA-2051: -- This was already addressed in

[jira] [Resolved] (KAFKA-2051) Add hashCode() and equals() to ProducerRecord

[ https://issues.apache.org/jira/browse/KAFKA-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2051. -- Resolution: Duplicate > Add hashCode() and equals() to ProducerRecord >

[jira] [Commented] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381337#comment-14381337 ] Jay Kreps commented on KAFKA-2050: -- Nice catch. > Avoid calling .size() on java.util.Con

Re: Plan of Controlled Shutdown

Removed the mention of the controlled shutdown tool from the wiki since we don't support it anymore - https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown On Wed, Mar 25, 2015 at 3:07 PM, Guozhang Wang wrote: > Mingjie, > > Controlled shutdown

Re: [DISCUSS] KIP-11- Authorization design for kafka security

Parth, We can make some 15 mins or so to discuss this at the next KIP hangout. Thanks, Neha On Wed, Mar 25, 2015 at 1:07 PM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > Hi all, > > I have modified the KIP to reflect the recent change request from the > reviewers. I have been workin

Re: Metrics package discussion

Aditya, If we are doing a deep dive, one of the things to investigate would be memory/GC performance. IIRC, when I was looking into codahale at LinkedIn, I remember it having quite a few memory management and GC issues while using histograms. In comparison, histograms in the new metrics package ar

[jira] [Created] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

Alex Jiao Ziheng created KAFKA-2052: --- Summary: zookeeper.connect does not work when specifying multiple zk nodes with chroot Key: KAFKA-2052 URL: https://issues.apache.org/jira/browse/KAFKA-2052 Pr

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

Re: Metrics package discussion

Here was my understanding of the issue last time. The yammer metrics use a random sample of requests to estimate the histogram. This allocates a fairly large array of longs (their values are longs rather than floats). A reasonable sample might be 8k entries which would give about 64KB per histogra

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when specifying multiple zk nodes with chroot

[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Jiao Ziheng updated KAFKA-2052: Description: I wish to specify multiple zk nodes with chroot strings in server.properties c

Re: Review Request 31806: Patch for KAFKA-1501

> On March 12, 2015, 1:04 p.m., Guozhang Wang wrote: > > core/src/test/scala/unit/kafka/utils/TestUtils.scala, line 67 > > > > > > The name is a bit misleading since the port value is actually fixed. > > Maybe we can r

Re: Review Request 31806: Patch for KAFKA-1501

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31806/ --- (Updated March 25, 2015, 7:45 a.m.) Review request for kafka. Bugs: KAFKA-150

[jira] [Commented] (KAFKA-1501) transient unit tests failures due to port already in use

[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379441#comment-14379441 ] Ewen Cheslack-Postava commented on KAFKA-1501: -- Updated reviewboard https://r

[jira] [Updated] (KAFKA-1501) transient unit tests failures due to port already in use

[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-1501: - Attachment: KAFKA-1501_2015-03-25_00:44:50.patch > transient unit tests failures d

[jira] [Commented] (KAFKA-1501) transient unit tests failures due to port already in use

[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379451#comment-14379451 ] Ewen Cheslack-Postava commented on KAFKA-1501: -- [~guozhang] Sorry, seems I mi

[jira] [Commented] (KAFKA-2038) Unable to restart brokers after it went down with no space left on disk

[ https://issues.apache.org/jira/browse/KAFKA-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379515#comment-14379515 ] K Zakee commented on KAFKA-2038: On Mar 21, 2015, at 11:50 AM, svante karlsson wrote: Th

[jira] [Commented] (KAFKA-1982) change kafka.examples.Producer to use the new java producer

[ https://issues.apache.org/jira/browse/KAFKA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379898#comment-14379898 ] Ashish K Singh commented on KAFKA-1982: --- Pinging [~junrao] again :) > change kafka.

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379900#comment-14379900 ] Ashish K Singh commented on KAFKA-1856: --- Thanks for helping out [~charmalloc]! > Ad

[jira] [Commented] (KAFKA-1994) Evaluate performance effect of chroot check on Topic creation

[ https://issues.apache.org/jira/browse/KAFKA-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379959#comment-14379959 ] Ashish K Singh commented on KAFKA-1994: --- [~gwenshap] My apologies for the delay. Kin

Re: Review Request 30809: Patch for KAFKA-1888

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30809/#review77732 --- bin/kafka-run-class.sh

Metrics package discussion

Hey everyone, Picking up this discussion after yesterdays KIP hangout. For anyone who did not join the meeting, we have 2 different metrics packages being used by the clients (custom package) and the server (codahale). We are discussing whether to migrate the server to the new package. What in

Re: [VOTE] KIP-15 add a close with timeout to new producer

+1 -Jay On Tue, Mar 24, 2015 at 2:43 PM, Guozhang Wang wrote: > +1. > > On Tue, Mar 24, 2015 at 2:15 PM, Jiangjie Qin > wrote: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-15+-+Add+a+close+method+with+a+timeout+in+the+producer > > > > As a short summary, the new interface

Re: [VOTE] KIP-15 add a close with timeout to new producer

Wait, actually, why would the thread block forever? I would understand throwing an error and failing immediately (fail fast) and I would understand logging an error and blocking for the time they specified (since that is what they asked for), but the logging an error and putatively blocking forever

[jira] [Commented] (KAFKA-2038) Unable to restart brokers after it went down with no space left on disk

[ https://issues.apache.org/jira/browse/KAFKA-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380190#comment-14380190 ] Mayuresh Gharat commented on KAFKA-2038: I suppose this will involve manual interv

Re: Review Request 31369: Patch for KAFKA-1982

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31369/#review77747 --- examples/src/main/java/kafka/examples/Producer.java

Re: MirrorMaker improvements

To amortize the long RTT across data centers, you can tune the TCP window size by configuring a larger socket.receive.buffer.bytes in the consumer. For the last one, it seems that you want identical mirroring. The tricky thing is to figure out how to avoid duplicates when there is a failure. We ha

Re: Review Request 31369: Patch for KAFKA-1982

> On March 25, 2015, 4:48 p.m., Mayuresh Gharat wrote: > > examples/src/main/java/kafka/examples/Producer.java, line 56 > > > > > > messageNo++ can be outside the if-else block and used once. LGTM else. - Mayuresh

Re: [VOTE] KIP-15 add a close with timeout to new producer

Hi Jay, The reason we keep the blocking behavior if close() or close(timeout) is called from callback is discussed in another thread. I copy/paste the message here: It looks that the problem we want to solve and the purpose we want to achieve is: If user uses close() in callback, we want to let u

Re: MirrorMaker improvements

Hi Jun, I think bumping up socket.receive.buffer.bytes will help but might not be sufficient. There are actually two related factors here: 1. Pipelining TCP packets when send a single request/response. 2. Pipelining multiple requests/responses Bumping up socket.receive.buffer.bytes help with the 1)

Re: Review Request 31806: Patch for KAFKA-1501

> On March 12, 2015, 1:04 p.m., Guozhang Wang wrote: > > core/src/test/scala/unit/kafka/utils/TestUtils.scala, line 67 > > > > > > The name is a bit misleading since the port value is actually fixed. > > Maybe we can r

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380359#comment-14380359 ] Jiangjie Qin commented on KAFKA-1660: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 31850: Patch for KAFKA-1660

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated March 25, 2015, 5:55 p.m.) Review request for kafka. Bugs: KAFKA-166

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660_2015-03-25_10:55:42.patch > Ability to call close() with a timeout on the Jav

Plan of Controlled Shutdown

Hi. I've been trying to figure out the best way to do kafka broker rolling restart, and read the controlled shutdown wiki page: https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown However I cannot find this kafka.admin.ShutdownBroker class in

Re: Review Request 32440: Patch for KAFKA-2043

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32440/ --- (Updated March 25, 2015, 6:29 p.m.) Review request for kafka. Bugs: KAFKA-204

[jira] [Updated] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

[ https://issues.apache.org/jira/browse/KAFKA-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2043: --- Attachment: KAFKA-2043_2015-03-25_13:28:52.patch > CompressionType is passed in each RecordAccumulator

[jira] [Commented] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

[ https://issues.apache.org/jira/browse/KAFKA-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380433#comment-14380433 ] Grant Henke commented on KAFKA-2043: Updated reviewboard https://reviews.apache.org/r/

Re: Review Request 31816: Fix for KAFKA-527

> On March 21, 2015, 1:42 a.m., Jun Rao wrote: > > core/src/main/scala/kafka/message/ByteBufferMessageSet.scala, line 208 > > > > > > Can we just use wrapperMessage directly? I think we can, as we are just accessing i

kafka system tests

Hi dev list, I've been discussing the current state of system tests with Jun and others, and have summarized goals moving forward at: https://cwiki.apache.org/confluence/display/KAFKA/System+Test+Improvements Feedback is welcome! Thanks, Geoff

Re: Review Request 31816: Fix for KAFKA-527

> On March 21, 2015, 1:42 a.m., Jun Rao wrote: > > core/src/main/scala/kafka/message/ByteBufferMessageSet.scala, lines 241-243 > > > > > > In the normal path, this should only happen when reading the offset. So > > we

Re: Review Request 31742: Patch for KAFKA-527

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31742/ --- (Updated March 25, 2015, 7:08 p.m.) Review request for kafka. Bugs: KAFKA-527

Re: kafka system tests

Thanks for summarizing! I think we are all feeling the pain here and want to make life easier moving forward. The framework discussion is particularly interesting - unfortunately, the link to ducttape is broken at the moment. Gwen On Wed, Mar 25, 2015 at 11:46 AM, Geoffrey Anderson wrote: > Hi

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-527: --- Attachment: KAFKA-527_2015-03-25_12:08:00.patch > Compression support does numerous byte copies

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380547#comment-14380547 ] Yasuhiro Matsuda commented on KAFKA-527: Updated reviewboard https://reviews.apache

Re: C++ Client Library -- libkafka-asio

Hi, Thanks for your comment and thank you for adding the library to the wiki page. I haven't done any measurements regarding the performance of the library, yet. This, plus completing the documentation, are the missing parts before version 0.1.0. Of course, compression also needs to be added at s

Producer Behavior When one or more Brokers' Disk is Full.

Hello Kafka Community, What is expected behavior on Producer side when one or more Brokers’ disk is full, but have not reached retention period for topics (by size or by time limit). Does producer send data to that particular brokers and/or Producer Queue gets full and always throws Queue F

Re: Plan of Controlled Shutdown

Hi ,        you can enable/disable controlled shutdown using controlled.shutdown.enable in server.properties. This is by default set to “true” in 0.8.2. You can go ahead do a rolling restart  and you don’t need ShutdownBroker command which is removed.  Regarding KAFKA-2029 talks about improveme

Re: kafka system tests

Hi Gwen, Sorry about that, the ducttape repository was not yet public, but now it is. Cheers, Geoff On Wed, Mar 25, 2015 at 12:08 PM, Gwen Shapira wrote: > Thanks for summarizing! I think we are all feeling the pain here and want > to make life easier moving forward. > > The framework discussi

Re: [DISCUSS] KIP-11- Authorization design for kafka security

Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380657#comment-14380657 ] Jun Rao commented on KAFKA-527: --- [~ymatsuda], thanks for the patch. +1 and committed to trunk

Re: Review Request 32440: Patch for KAFKA-2043

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32440/#review77787 --- Ship it! - Mayuresh Gharat On March 25, 2015, 6:29 p.m., Grant He

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380693#comment-14380693 ] Guozhang Wang commented on KAFKA-527: - Updated reviewboard https://reviews.apache.org/r

Re: Review Request 31816: Fix decompression regarding KAFKA-572

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31816/ --- (Updated March 25, 2015, 8:26 p.m.) Review request for kafka. Summary (update

Re: Review Request 31967: Patch for KAFKA-1546

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31967/ --- (Updated March 25, 2015, 8:27 p.m.) Review request for kafka. Bugs: KAFKA-154

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380696#comment-14380696 ] Aditya A Auradkar commented on KAFKA-1546: -- Updated reviewboard https://reviews.a

[jira] [Updated] (KAFKA-1546) Automate replica lag tuning

[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-1546: - Attachment: KAFKA-1546_2015-03-25_13:27:40.patch > Automate replica lag tuning > -

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380706#comment-14380706 ] Jay Kreps commented on KAFKA-527: - Yeah it would be great to get the [~guozhang]'s patch in

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380718#comment-14380718 ] Guozhang Wang commented on KAFKA-2045: -- Agreed, and I remember we did have some discu

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-527: Attachment: KAFKA-527_2015-03-25_13:26:36.patch > Compression support does numerous byte copies > ---

Re: Review Request 32465: Patch for KAFKA-2047

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32465/#review77789 --- core/src/main/scala/kafka/tools/MirrorMaker.scala

Re: Review Request 32465: Patch for KAFKA-2047

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32465/ --- (Updated March 25, 2015, 8:48 p.m.) Review request for kafka. Bugs: KAFKA-204

[jira] [Commented] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380737#comment-14380737 ] Jiangjie Qin commented on KAFKA-2047: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 32465: Patch for KAFKA-2047

> On March 25, 2015, 8:37 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/tools/MirrorMaker.scala, line 325 > > > > > > Wondering if we call System.exit(-1) directly from one of the MM > > thread, if we have

[jira] [Updated] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2047: Attachment: KAFKA-2047_2015-03-25_13:48:34.patch > Accelarate consumer rebalance in mirror maker. >

Re: Review Request 32465: Patch for KAFKA-2047

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32465/#review77791 --- Ship it! Will make the minor change upon checkin. core/src/main/s

Re: Review Request 32465: Patch for KAFKA-2047

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32465/#review77794 --- core/src/main/scala/kafka/tools/MirrorMaker.scala

[jira] [Updated] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2047: - Resolution: Fixed Status: Resolved (was: Patch Available) > Accelarate consumer rebalance

[jira] [Commented] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380758#comment-14380758 ] Guozhang Wang commented on KAFKA-2047: -- Thanks for the patch, +1 and committed to tru

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380776#comment-14380776 ] Onur Karaman commented on KAFKA-2046: - It looks like I hit a deadlock yesterday betwee

[jira] [Created] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

Tim Brooks created KAFKA-2050: - Summary: Avoid calling .size() on java.util.ConcurrentLinkedQueue Key: KAFKA-2050 URL: https://issues.apache.org/jira/browse/KAFKA-2050 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Brooks updated KAFKA-2050: -- Attachment: dont_call_queue_size.patch > Avoid calling .size() on java.util.ConcurrentLinkedQueue >

Build failed in Jenkins: Kafka-trunk #433

See Changes: [junrao] kafka-527; Compression support does numerous byte copies; patched by Yasuhiro Matsuda; reviewed by Guozhang Wang and Jun Rao [wangguoz] KAFKA-2047; Move the stream creation into concurrent mirror maker threads; revie

Re: Review Request 31967: Patch for KAFKA-1546

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31967/#review77799 --- core/src/main/scala/kafka/cluster/Replica.scala

[jira] [Commented] (KAFKA-2050) Avoid calling .size() on java.util.ConcurrentLinkedQueue

[ https://issues.apache.org/jira/browse/KAFKA-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380809#comment-14380809 ] Sriharsha Chintalapani commented on KAFKA-2050: --- [~timbrooks] changes looks

Jenkins build is back to normal : KafkaPreCommit #45

See

Re: Review Request 32459: Patch for KAFKA-2044

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32459/#review77805 --- Thanks for the patch. Looks good to me overall. A few comments below

Re: Review Request 32459: Patch for KAFKA-2044

> On March 25, 2015, 9:43 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/requests/AbstractRequestResponse.java, > > lines 46-54 > > > > > > Instead of adding this method here, would it be better

[jira] [Commented] (KAFKA-2038) Unable to restart brokers after it went down with no space left on disk

[ https://issues.apache.org/jira/browse/KAFKA-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380850#comment-14380850 ] K Zakee commented on KAFKA-2038: Thanks, I was able to start the broker after manually mov

Re: Review Request 31606: Patch for KAFKA-1416

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31606/#review77804 --- core/src/test/scala/unit/kafka/integration/FetcherTest.scala

Re: Review Request 32459: Patch for KAFKA-2044

> On March 25, 2015, 9:43 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/requests/AbstractRequestResponse.java, > > lines 46-54 > > > > > > Instead of adding this method here, would it be better

Re: MirrorMaker improvements

@Guozhang We actually have separate topics depending on the source of the message and the multicast distribution group (the set of destinations). Our topics are named: source_multicast-group. We do not aggregate data but we do static routing based on the destination and the destination set (that is

  1   2   >