Re: Review Request 20997: Fix KAFKA-1432

> On May 2, 2014, 1:35 a.m., Jun Rao wrote: > > perf/src/main/scala/kafka/perf/ProducerPerformance.scala, lines 308-309 > > > > > > Is there a reason that we change to send a null key? Will that break > > system tests

Re: Review Request 20997: Fix KAFKA-1432

> On May 2, 2014, 5:51 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/producer/TransitProducer.scala, lines 24-25 > > > > > > +1 on renaming this as per Jun's suggestion. > > > > If this is only for t

Re: Review Request 20745: Patch for KAFKA-1397

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745/#review42058 --- core/src/main/scala/kafka/controller/TopicDeletionManager.scala

Re: Review Request 20997: Fix KAFKA-1432: address Jun's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/ --- (Updated May 5, 2014, 5:31 p.m.) Review request for kafka. Summary (updated)

[jira] [Updated] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1432: - Attachment: KAFKA-1432_2014-05-05_10:31:42.patch > Make num.producerThreads configurable on new M

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/#review42169 --- core/src/main/scala/kafka/server/BrokerStates.scala

[jira] [Commented] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989743#comment-13989743 ] Guozhang Wang commented on KAFKA-1432: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/ --- (Updated May 5, 2014, 6:05 p.m.) Review request for kafka. Bugs: KAFKA-1384

[jira] [Updated] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-05_11:05:15.patch > Log Broker state > - > >

[jira] [Commented] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989785#comment-13989785 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/#review41974 --- clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.ja

Re: Review Request 20745: Patch for KAFKA-1397

> On May 5, 2014, 5:12 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/utils/ShutdownableThread.scala, lines 28-33 > > > > > > I thought you plan to remove isShuttingDown? Odd I don't have this anymore in my box, bu

Re: Review Request 20745: Patch for KAFKA-1397

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745/ --- (Updated May 5, 2014, 6:17 p.m.) Review request for kafka. Bugs: KAFKA-1397

[jira] [Commented] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989793#comment-13989793 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-05_11:17:59.patch > delete topic is not working > -

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/ --- (Updated May 5, 2014, 6:35 p.m.) Review request for kafka. Bugs: KAFKA-1328

[jira] [Commented] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989817#comment-13989817 ] Neha Narkhede commented on KAFKA-1328: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/ --- (Updated May 5, 2014, 6:35 p.m.) Review request for kafka. Bugs: KAFKA-1328

[jira] [Updated] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1328: - Attachment: KAFKA-1328_2014-05-05_11:35:41.patch > Add new consumer APIs > -

[jira] [Commented] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989818#comment-13989818 ] Neha Narkhede commented on KAFKA-1328: -- Updated reviewboard https://reviews.apache.or

[jira] [Updated] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1328: - Attachment: KAFKA-1328_2014-05-05_11:35:07.patch > Add new consumer APIs > -

Re: Review Request 20997: Fix KAFKA-1432: address Jun's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/#review42185 --- core/src/main/scala/kafka/producer/BaseProducer.scala

Re: Review Request 20997: Fix KAFKA-1432: address Jun's comments

> On May 2, 2014, 5:51 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/producer/ConsoleProducer.scala, line 50 > > > > > > It will be cleaner to move this logic into a buildProducerConfig > > private API that

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (86 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1432 Make num.producerThreads configurable on new MirrrorMaker https://issues.apache.org/jira/browse/KAFKA-1432 KAFKA-1397 d

Re: Review Request 20745: Patch for KAFKA-1397

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745/#review42195 --- core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/#review42197 --- Compilation error when trying to run the unit tests. kafka_gradle/c

Re: Review Request 20745: Patch for KAFKA-1397

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745/#review42199 --- core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala

Re: Review Request 20745: Patch for KAFKA-1397

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745/ --- (Updated May 5, 2014, 9 p.m.) Review request for kafka. Bugs: KAFKA-1397

[jira] [Commented] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989968#comment-13989968 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-05_14:00:29.patch > delete topic is not working > -

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/ --- (Updated May 5, 2014, 9:25 p.m.) Review request for kafka. Bugs: KAFKA-1384

Re: Review Request 20718: Patch for KAFKA-1384

> On May 5, 2014, 8:51 p.m., Jun Rao wrote: > > Compilation error when trying to run the unit tests. > > > > kafka_gradle/core/src/test/scala/unit/kafka/server/HighwatermarkPersistenceTest.scala:35: > > not enough arguments for constructor LogManager: (logDirs: > > Array[java.io.File],topicCon

[jira] [Commented] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989994#comment-13989994 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-05_14:25:25.patch > Log Broker state > - > >

Re: Review Request 20997: Fix KAFKA-1432: address Jun's comments

> On May 2, 2014, 5:51 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/producer/TransitProducer.scala, lines 24-25 > > > > > > +1 on renaming this as per Jun's suggestion. > > > > If this is only for t

Re: Review Request 20997: Fix KAFKA-1432: address Neha's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/ --- (Updated May 5, 2014, 9:33 p.m.) Review request for kafka. Summary (updated)

[jira] [Updated] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1432: - Attachment: KAFKA-1432_2014-05-05_14:33:01.patch > Make num.producerThreads configurable on new M

[jira] [Commented] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989997#comment-13989997 ] Guozhang Wang commented on KAFKA-1432: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 20997: Fix KAFKA-1432: address Neha's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/#review42212 --- core/src/main/scala/kafka/tools/MirrorMaker.scala

[jira] [Commented] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990052#comment-13990052 ] Guozhang Wang commented on KAFKA-1432: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 20997: Fix KAFKA-1432: address Neha's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/ --- (Updated May 5, 2014, 10:24 p.m.) Review request for kafka. Bugs: KAFKA-1432

[jira] [Updated] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1432: - Attachment: KAFKA-1432_2014-05-05_15:24:08.patch > Make num.producerThreads configurable on new M

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/#review42218 --- core/src/test/scala/unit/kafka/log/LogManagerTest.scala

Re: Review Request 20997: Fix KAFKA-1432: address Neha's comments

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997/#review42221 --- Ship it! Ship It! - Neha Narkhede On May 5, 2014, 10:24 p.m., Gu

[jira] [Resolved] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1432. -- Resolution: Fixed Thanks for the patches, Guozhang. Pushed to trunk > Make num.producerThreads

[jira] [Updated] (KAFKA-1431) ConsoleConsumer - Option to clean zk consumer path

[ https://issues.apache.org/jira/browse/KAFKA-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sriharsha chintalapani updated KAFKA-1431: -- Attachment: KAFKA-1431-v1.patch > ConsoleConsumer - Option to clean zk consumer

[jira] [Updated] (KAFKA-1431) ConsoleConsumer - Option to clean zk consumer path

[ https://issues.apache.org/jira/browse/KAFKA-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sriharsha chintalapani updated KAFKA-1431: -- Status: Patch Available (was: Open) > ConsoleConsumer - Option to clean zk con

[jira] [Commented] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990123#comment-13990123 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1384) Log Broker state

[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-05_17:14:57.patch > Log Broker state > - > >

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/ --- (Updated May 6, 2014, 12:15 a.m.) Review request for kafka. Bugs: KAFKA-1384

Re: Review for the new consumer APIs

Hi Neha, How will new Consumer help us with implementing following use case? We have heartbeat as one of topics and all application servers publish metric to this topic. We have to meet near real-time consume SLA (less than 30 seconds). 1) We would like to find out what is latest message per

Re: Review Request 20718: Patch for KAFKA-1384

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20718/#review42228 --- Ship it! Minor comment, mentioned earlier: prefer just State instea

[jira] [Commented] (KAFKA-1273) Brokers should make sure replica.fetch.max.bytes >= message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990162#comment-13990162 ] sriharsha chintalapani commented on KAFKA-1273: --- I am not able to reproduce

QOS on Producer Side

We are using Kafka for operation metrics and we do not want to loose any data at all if there is issue with Network or all brokers needs to be rebooted or operation reason down time while all producers are still producing data on front end side. We use async to publish messages and we are using Ka

Re: QOS on Producer Side

At LinkedIn, we only buffer messages in memory in the producer. We try as much as we can to make the Kafka cluster always available (with replication). The "on failure" feature you mentioned can be implemented by providing a callback. You can see the example in the javadoc. *send

Re: QOS on Producer Side

Thanks for answers. Does the callback get call on failure only or for success as well ? Also, how do I do this on Kafka 0.8.0 ? Is there any plan for adding buffering on disk for next version ? Also, when application restart, kafka producer will have to transmit the messages (buffered message