Re: Make kafka storage engine pluggable and provide a HDFS plugin?

Hi Steve, Yes, what I want is that Kafka doesn't have to care about machines physically (as an option). Best, Hangjun 2014-05-21 11:46 GMT+08:00 Steve Morin : > Hangjun, > Does having Kafka in Yarn would be a big architectural change from where > it is now? From what I have seen on most typi

[jira] [Resolved] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1328. -- Resolution: Fixed Pushed the javadoc updates as well. > Add new consumer APIs > --

Re: Make kafka storage engine pluggable and provide a HDFS plugin?

Hangjun, Does having Kafka in Yarn would be a big architectural change from where it is now? From what I have seen on most typical setup you want machines optimized for Kafka, not just it on top of hdfs. -Steve On Tue, May 20, 2014 at 8:37 PM, Hangjun Ye wrote: > Thanks Jun and Francois. > >

Re: Make kafka storage engine pluggable and provide a HDFS plugin?

Thanks Jun and Francois. We used Kafka 0.8.0 previously. We got some weird error when expanding cluster and it couldn't be finished. Now we use 0.8.1.1, I would have a try on cluster expansion sometime. I read the discussion on that jira issue and I agree with points raised there. HDFS was also i

Jenkins build is back to normal : Kafka-trunk #188

See

Re: Review Request 21744: Patch for KAFKA-1446

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21744/#review43564 --- This looks good. One minor issue with the rebalance time metric in t

[jira] [Commented] (KAFKA-1446) Consumer metrics for rebalance

[ https://issues.apache.org/jira/browse/KAFKA-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004180#comment-14004180 ] Sriharsha Chintalapani commented on KAFKA-1446: --- [~nehanarkhede] [~clarkhask

[jira] [Commented] (KAFKA-1446) Consumer metrics for rebalance

[ https://issues.apache.org/jira/browse/KAFKA-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004167#comment-14004167 ] Sriharsha Chintalapani commented on KAFKA-1446: --- Created reviewboard https:/

[jira] [Updated] (KAFKA-1446) Consumer metrics for rebalance

[ https://issues.apache.org/jira/browse/KAFKA-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1446: -- Attachment: KAFKA-1446.patch > Consumer metrics for rebalance > ---

Review Request 21744: Patch for KAFKA-1446

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21744/ --- Review request for kafka. Bugs: KAFKA-1446 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004136#comment-14004136 ] Neha Narkhede commented on KAFKA-1328: -- Checked in the latest patch. Will keep this J

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/#review43558 --- Ship it! clients/src/main/java/org/apache/kafka/clients/consumer/C

[jira] [Updated] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1328: - Attachment: KAFKA-1328_2014-05-20_16:34:37.patch > Add new consumer APIs > -

[jira] [Commented] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004127#comment-14004127 ] Neha Narkhede commented on KAFKA-1328: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/ --- (Updated May 20, 2014, 11:34 p.m.) Review request for kafka. Bugs: KAFKA-1328

[jira] [Commented] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004094#comment-14004094 ] Neha Narkhede commented on KAFKA-1328: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 19731: Patch for KAFKA-1328

> On May 19, 2014, 6:51 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java, > > lines 27-28 > > > > > > Have we added the config for selecting the strategy for assigni

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/ --- (Updated May 20, 2014, 10:55 p.m.) Review request for kafka. Bugs: KAFKA-1328

[jira] [Updated] (KAFKA-1328) Add new consumer APIs

[ https://issues.apache.org/jira/browse/KAFKA-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1328: - Attachment: KAFKA-1328_2014-05-20_15:55:01.patch > Add new consumer APIs > -

Re: Review Request 19731: Patch for KAFKA-1328

> On May 19, 2014, 6:22 p.m., Guozhang Wang wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java, > > lines 99-119 > > > > > > Should we discuss about code conventions of using ellipse (i.e

Re: Review Request 19731: Patch for KAFKA-1328

> On May 19, 2014, 5:30 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java, > > lines 70-71 > > > > > > Instead of introducing ConsumerRecords, I'd prefer just returning >

[jira] [Commented] (KAFKA-1316) Refactor Sender

[ https://issues.apache.org/jira/browse/KAFKA-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004009#comment-14004009 ] Jay Kreps commented on KAFKA-1316: -- I started on this and I think I have a design that at

[jira] [Updated] (KAFKA-1462) Add new request and response formats for the new consumer and coordinator communication

[ https://issues.apache.org/jira/browse/KAFKA-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1462: - Fix Version/s: 0.9.0 > Add new request and response formats for the new consumer and coordinator

[jira] [Created] (KAFKA-1462) Add new request and response formats for the new consumer and coordinator communication

Guozhang Wang created KAFKA-1462: Summary: Add new request and response formats for the new consumer and coordinator communication Key: KAFKA-1462 URL: https://issues.apache.org/jira/browse/KAFKA-1462

[jira] [Commented] (KAFKA-1449) Extend wire protocol to allow CRC32C

[ https://issues.apache.org/jira/browse/KAFKA-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003736#comment-14003736 ] Jun Rao commented on KAFKA-1449: If we need to change the attribute or the magic in the me

[jira] [Updated] (KAFKA-1459) kafka.tools.ConsumerOffsetChecker throws NoNodeException

[ https://issues.apache.org/jira/browse/KAFKA-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1459: --- Labels: newbie (was: ) > kafka.tools.ConsumerOffsetChecker throws NoNodeException >

[jira] [Commented] (KAFKA-1458) kafka hanging on shutdown

[ https://issues.apache.org/jira/browse/KAFKA-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003725#comment-14003725 ] Jun Rao commented on KAFKA-1458: Could you try 0.8.1.1, which fixed some deadlock issues d

Re: Review Request 21588: Fix KAFKA-1430

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21588/#review43423 --- The patch looks promising. The places that we make a call to unblock

Review Request 21712: Patch for KAFKA-1444

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21712/ --- Review request for kafka. Bugs: KAFKA-1444 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1444) kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the toString method

[ https://issues.apache.org/jira/browse/KAFKA-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1444: -- Attachment: KAFKA-1444.patch > kafka.javaapi.TopicMetadata and PartitionMetadat

[jira] [Commented] (KAFKA-1444) kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the toString method

[ https://issues.apache.org/jira/browse/KAFKA-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003643#comment-14003643 ] Sriharsha Chintalapani commented on KAFKA-1444: --- Created reviewboard https:/

Re: Make kafka storage engine pluggable and provide a HDFS plugin?

Take a look at Camus François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure Capitaine Club Capra VP-Communication - CS Games 2014 Jeux de Génie

Build failed in Jenkins: Kafka-trunk #187

See Changes: [jjkoshy] KAFKA-1437; Consumer metadata response should include (empty) coordinator information if the coordinator is unavailable; reviewed by Neha Narkhede and Guozhang Wang. [neha.narkhede] KAFKA-1431 ConsoleConsumer - Opti