kafka 0.8 beta release status

We have updated the 0.8 documentation in our website ( http://kafka.apache.org/index.html). Please review the docs. We have the following blockers for the 0.8 beta release: additional docs: * examples of using the 0.8 high level consumer api * description of additional 0.8 tools KAFKA-885 (sbt pa

Re: Kafka wiki Documentation conventions - looking for feedback

Chris, Any update of the high level consumer example? Also, in the Producer example, it would be useful to describe how to write a customized encoder. One subtle thing is that the encoder needs a constructor that takes a a single VerifiableProperties argument ( https://issues.apache.org/jira/brow

Re: 0.7.2 Async Producer Question

Yes, that's the right way to block on queue full. Thanks, Jun On Sun, Apr 28, 2013 at 12:30 PM, Suyog Rao wrote: > Hi, in Kafka 0.7.2 we are getting QueueFullExceptions while using the > AsyncProducer with queue.size = 50K and 1 producer. I read that we can make > this internal queue blocking

We have renamed certain Zookeeper and Producer configs

Hello everyone, If you update your Kafka 0.8 to HEAD, please note that we have renamed the following configs (https://issues.apache.org/jira/browse/KAFKA-871): On the broker and consumer: * zk.connect has been renamed to zookeeper.connect * zk.session.timeout.ms has been renamed to zook

0.7.2 Async Producer Question

Hi, in Kafka 0.7.2 we are getting QueueFullExceptions while using the AsyncProducer with queue.size = 50K and 1 producer. I read that we can make this internal queue blocking by setting queue.enqueue.timeout.ms = -1. Is that possible in 0.7.2? On the broker side, the log.flush.interval is set to

Re: kafka --> hadoop question

Sounds good :-) On Sun, Apr 28, 2013 at 8:24 PM, Neha Narkhede wrote: > You can ask that question on the camus mailing list. > > Thanks, > Neha > On Apr 28, 2013 10:14 AM, "Oleg Ruchovets" wrote: > > > Thank you Neha. > > Is Camus stable enough? Is anybody already (except LinkedIn :-) ) > u

Re: kafka --> hadoop question

You can ask that question on the camus mailing list. Thanks, Neha On Apr 28, 2013 10:14 AM, "Oleg Ruchovets" wrote: > Thank you Neha. > Is Camus stable enough? Is anybody already (except LinkedIn :-) ) used > in in production? > Thanks > Oleg. > > > On Sun, Apr 28, 2013 at 5:17 PM, Neha Nark

Re: kafka --> hadoop question

Thank you Neha. Is Camus stable enough? Is anybody already (except LinkedIn :-) ) used in in production? Thanks Oleg. On Sun, Apr 28, 2013 at 5:17 PM, Neha Narkhede wrote: > Camus is a solution that is well supported and currently used at LinkedIn. > > Thanks, > Neha > On Apr 28, 2013 4:30 A

Re: kafka --> hadoop question

Camus is a solution that is well supported and currently used at LinkedIn. Thanks, Neha On Apr 28, 2013 4:30 AM, "Oleg Ruchovets" wrote: > Hi , >I am looking for simple way to transfer from kafka to hadoop. > >I found such solutions in Github: > https://github.com/linkedin/camus >

kafka --> hadoop question

Hi , I am looking for simple way to transfer from kafka to hadoop. I found such solutions in Github: https://github.com/linkedin/camus https://github.com/miniway/kafka-hadoop-consumer https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer Question: What

Re: Kafka broker not respecting log.roll.hours?

@Dan: Upon restart of the broker, if a segment already has data, the broker resets the firstAppendTime of the segment to the time when that segment's file handles are being loaded into memory. Thus as you correctly explained, every time you shut down a broker, the broker essentially forgets the fir