Kafka not showing log/lag size even though there is data in logs

2014-01-19 Thread Gaurav Sehgal
I am using Kafka version 0.8, I have data in Kafka logs and wanted to replay that so I used the kafka tool to reset consumer offsetsto 0. Though now when I run the ConsumerOffsetChecker tool does not show me anything, even though there is data in the logs. The following is the output from the too

Re: log.retention.bytes.per.topic does not trigger deletion

2014-01-19 Thread Joe Stein
There are a lot of nuance changes to the QuickStart and other changes for 0.8.1 that will need a separate page... Not just modifications. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @al

Which properties should I sync if I want to increase message.max.bytes?

2014-01-19 Thread Bae, Jae Hyeon
If I want to increase message.max.bytes up to 10MB *with compression*, Are there any properties I sync except the following two properties? - broker's message.max.bytes - consumer's fetch.message.max.bytes What about replica.message.max.bytes? Thank you Best, Jae

Question on MessageSizeTooLargeException

2014-01-19 Thread Bae, Jae Hyeon
Hello I finally upgraded kafka 0.7 to kafka 0.8 and a few kafka 0.8 clusters are being tested now. Today, I got alerted with the following messages: "data": { "exceptionMessage": "Found a message larger than the maximum fetch size of this consumer on topic nf_errors_log partition 0 at fetch

Re: log.retention.bytes.per.topic does not trigger deletion

2014-01-19 Thread Jason Rosenberg
Please be sure to update the online config docs with this change! The per topic options are still listed there Jason On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer wrote: > I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta > after it became the "stable version". > Good