Re: Kafka/Hadoop consumers and producers

We also have a need today to ETL from Kafka into Hadoop and we do not currently nor have any plans to use Avro. So is the official direction based on this discussion to ditch the Kafka contrib code and direct people to use Camus without Avro as Ken described or are both solutions going to surv

[jira] [Updated] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-990: - Attachment: KAFKA-990-v1-rebased.patch > Fix ReassignPartitionCommand and improve usab

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-992: Attachment: KAFKA-992.v6.patch 1.1-3. Done. > Double Check on Broker Registration t

[jira] [Commented] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733796#comment-13733796 ] Neha Narkhede commented on KAFKA-990: - Thanks for the rebased patch, Sriram. Overall, t

[jira] [Comment Edited] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733796#comment-13733796 ] Neha Narkhede edited comment on KAFKA-990 at 8/8/13 6:16 PM: - T

[jira] [Commented] (KAFKA-990) Fix ReassignPartitionCommand and improve usability

[ https://issues.apache.org/jira/browse/KAFKA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733837#comment-13733837 ] Joel Koshy commented on KAFKA-990: -- - Topics to move json file format seems unnecessarily

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (66 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-998 Producer should not retry on non-recoverable error codes https://issues.apache.org/jira/browse/KAFKA-998 KAFKA-9

[jira] [Created] (KAFKA-1004) Handle topic event for trivial whitelist topic filters

Guozhang Wang created KAFKA-1004: Summary: Handle topic event for trivial whitelist topic filters Key: KAFKA-1004 URL: https://issues.apache.org/jira/browse/KAFKA-1004 Project: Kafka Issue Ty

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734114#comment-13734114 ] Neha Narkhede commented on KAFKA-992: - Thanks for the follow up patch Guozhang. Overall

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-992: Attachment: KAFKA-992.v7.patch Thanks for the comment Neha. 9.1: Done. 9.2,3 Done. 10.1,2,3: Done

Re: Kafka/Hadoop consumers and producers

The contrib code is simple and probably wouldn't require too much work to fix, but it's a lot less robust than Camus, so you would ideally need to do some work to make it solid against all edge cases, failure scenarios and performance bottlenecks... I would definitely recommend investing in Camus

[jira] [Created] (KAFKA-1005) kafka.perf.ConsumerPerformance not shutting down consumer

paul mackles created KAFKA-1005: --- Summary: kafka.perf.ConsumerPerformance not shutting down consumer Key: KAFKA-1005 URL: https://issues.apache.org/jira/browse/KAFKA-1005 Project: Kafka Issue T

[jira] [Updated] (KAFKA-1005) kafka.perf.ConsumerPerformance not shutting down consumer

[ https://issues.apache.org/jira/browse/KAFKA-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated KAFKA-1005: Attachment: ConsumerPerformance.scala.patch > kafka.perf.ConsumerPerformance not shutting down

Re: Kafka/Hadoop consumers and producers

Felix, The Camus route is the direction I have headed for allot of the reasons that you described. The only wrinkle is we are still on Kafka 0.7.3 so I am in the process of back porting this patch: https://github.com/linkedin/camus/commit/87917a2aea46da9d21c8f67129f6463af52f7aa8 that is descri