Re: Compute offset for messages compressed with Snappy inKakfa 0.8.0

2014-03-19 Thread Jun Rao
Do you mean Camus didn't write any data to HDFS or you can't read the data on HDFS? SimpleConsumer works with compressed data. You just need to make sure that the fetch size is larger than the compressed set of messages. Also, have you asked the Camus mailing list? Thanks, Jun On Wed, Mar 19, 2

Compute offset for messages compressed with Snappy inKakfa 0.8.0

2014-03-19 Thread Zhu Wayne
I am using Camus to get compressed Snappy messages into HDFS. However, I can't get records in HDFS even though CamusJob was completed successfully. Getting uncompressed AVRO messages into HDFS is fine. According to Neha's post, the consumer should be * agnostic to compression.* http://geekmantra.w