OK, I can't resist a bit more detail about my favorite project :) Here's a good blog post on how to use the Flume-Kafka source, sink and channel: http://blog.cloudera.com/blog/2014/11/flafka-apache-flume-meets-apache-kafka-for-event-processing/
One of the main differences from Baniuyao's version is that we support committing offsets to zookeeper on every batch (rather than automatically) and batch sizes are configurable so you can tune throughput / latency / cpu utilization / zookeeper load, etc. Gwen On Fri, Dec 12, 2014 at 6:10 PM, Gwen Shapira <gshap...@cloudera.com> wrote: > I'd use the one in Flume trunk (it was far better tested) and we also > have Kafka-channel :) > And it works with 0.8.1. > > https://github.com/apache/flume/tree/trunk/flume-ng-sources/flume-kafka-source > https://github.com/apache/flume/tree/trunk/flume-ng-sinks/flume-ng-kafka-sink > https://github.com/apache/flume/tree/trunk/flume-ng-channels/flume-kafka-channel > > Gwen > > On Fri, Dec 12, 2014 at 5:33 PM, Jack Ye <jack...@jasper.com> wrote: >> Does this work for kafka 0.8.1? thank you >> >> https://github.com/baniuyao/flume-kafka >>
