Re: Setting up kafka consumer to send logs to s3

2013-06-26 Thread Glenn Nethercutt
I did the same search about a month ago. I forked this kafka-s3-consumer from QubitProducts/kafka-s3-consumer to upfit it for the 0.8 api changes. You'll have to futz with th

Re: Setting up kafka consumer to send logs to s3

2013-06-25 Thread Florin Trofin
Quick search for "kafka s3 consumer" brings up a bunch of Github projects. If you don't like any, I would write a kafka consumer in java that writes to s3. Probably less than 200 lines of code. F. On 6/25/13 1:50 PM, "Alan Everdeen" wrote: >In my application, I have logs that are sent as kafka

Setting up kafka consumer to send logs to s3

2013-06-25 Thread Alan Everdeen
In my application, I have logs that are sent as kafka messages, and I need a way to save these logs to an s3 bucket using a kafka sink. I was wondering what the recommended approach would be to accomplish this task, assuming I am using Kafka 0.8. Thank you for your time, Alan Everdeen