Re: error handling with high-level consumer

2015-02-05 Thread Steven Wu
Jun, we are already passing the retention period. so can't go back and do a DumpLogSegment. plus there are other factors make this exercise difficult: 1) this topic has very high traffic volume 2) we don't know the msg offset that is corrupted anyhow, it doesn't happen often. but can you advise p

Re: error handling with high-level consumer

2015-02-04 Thread Jun Rao
1) Does the corruption happen to console consumer as well? If so, could you run DumpLogSegment tool to see if the data is corrupted on disk? Thanks, Jun On Wed, Feb 4, 2015 at 9:55 AM, Steven Wu wrote: > Hi, > > We have observed these two exceptions with consumer *iterator.next()* > recently.

error handling with high-level consumer

2015-02-04 Thread Steven Wu
Hi, We have observed these two exceptions with consumer *iterator.next()* recently. want to ask how should we handle them properly. *1) CRC corruption* Message is corrupt (stored crc = 433657556, computed crc = 3265543163) I assume in this case we should just catch it and move on to the next msg