librdkafka 0.8.0 released

This announces the 0.8.0 release of librdkafka - The Apache Kafka client C library - now with 0.8 protocol support. Features: * Producer (~800K msgs/s) * Consumer (~3M msgs/s) * Compression (Snappy, gzip) * Proper failover and leader re-election support - no message is ever lost. * Configuration

Re: librdkafka 0.8.0 released

Thanks for sharing this! What is the message size for the throughput numbers stated below? Thanks, Neha On Nov 22, 2013 6:59 AM, "Magnus Edenhill" wrote: > This announces the 0.8.0 release of librdkafka - The Apache Kafka client C > library - now with 0.8 protocol support. > > Features: > * Prod

[jira] [Updated] (KAFKA-1136) Add subAppend in Log4jAppender for generic usage

[ https://issues.apache.org/jira/browse/KAFKA-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1136: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. +1. Committed to tru

[jira] [Resolved] (KAFKA-1103) Consumer uses two zkclients

[ https://issues.apache.org/jira/browse/KAFKA-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-1103. Resolution: Fixed Thanks for the patch. +1. Committed to trunk after removing the unnecessary change to lo

[jira] [Commented] (KAFKA-1103) Consumer uses two zkclients

[ https://issues.apache.org/jira/browse/KAFKA-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830186#comment-13830186 ] Joel Koshy commented on KAFKA-1103: --- I think the issue is that the patch-review tool com

[jira] [Created] (KAFKA-1142) Patch review tool should take diff with origin from last divergent point

Joel Koshy created KAFKA-1142: - Summary: Patch review tool should take diff with origin from last divergent point Key: KAFKA-1142 URL: https://issues.apache.org/jira/browse/KAFKA-1142 Project: Kafka

Review Request 15793: Patch for KAFKA-1142

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15793/ --- Review request for kafka. Bugs: KAFKA-1142 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1142) Patch review tool should take diff with origin from last divergent point

[ https://issues.apache.org/jira/browse/KAFKA-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830191#comment-13830191 ] Joel Koshy commented on KAFKA-1142: --- Created reviewboard against branch origin/trunk >

[jira] [Updated] (KAFKA-1142) Patch review tool should take diff with origin from last divergent point

[ https://issues.apache.org/jira/browse/KAFKA-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1142: -- Attachment: KAFKA-1142.patch > Patch review tool should take diff with origin from last divergent point

[jira] [Commented] (KAFKA-1142) Patch review tool should take diff with origin from last divergent point

[ https://issues.apache.org/jira/browse/KAFKA-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830193#comment-13830193 ] Joel Koshy commented on KAFKA-1142: --- I got: rbtools.api.errors.APIError: HTTP 502 (from

[jira] [Commented] (KAFKA-1142) Patch review tool should take diff with origin from last divergent point

[ https://issues.apache.org/jira/browse/KAFKA-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830200#comment-13830200 ] Joel Koshy commented on KAFKA-1142: --- Actually, it did post a diff: https://reviews.apach

[jira] [Created] (KAFKA-1143) Consumer should cache topic partition info

Guozhang Wang created KAFKA-1143: Summary: Consumer should cache topic partition info Key: KAFKA-1143 URL: https://issues.apache.org/jira/browse/KAFKA-1143 Project: Kafka Issue Type: Bug

Review Request 15805: Patch for KAFKA-1140

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15805/ --- Review request for kafka. Bugs: KAFKA-1140 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1140) Move the decoding logic from ConsumerIterator.makeNext to next

[ https://issues.apache.org/jira/browse/KAFKA-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1140: - Attachment: KAFKA-1140.patch > Move the decoding logic from ConsumerIterator.makeNext to next > -

[jira] [Commented] (KAFKA-1140) Move the decoding logic from ConsumerIterator.makeNext to next

[ https://issues.apache.org/jira/browse/KAFKA-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830373#comment-13830373 ] Guozhang Wang commented on KAFKA-1140: -- Created reviewboard https://reviews.apache.or

Re: Kafka/Hadoop consumers and producers

I agree with you. We are looking for a simple solution for data from Kafka to Hadoop. I have tried using Camus earlier (Non-Avro) and documentation is lacking to make it work correctly, as we do not need to introduce another component to the solution. In the meantime, can the Kafka Hadoop Consu