Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-11 Thread Philip O'Toole
Works for me. It's the least disruptive option. > On Feb 11, 2014, at 3:11 PM, Suyog Rao wrote: > > Actually, looking at the code, the consumer client code can also catch this > exception while iterating for messages. The fetcher thread inserts a special > message before dying, which triggers

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-11 Thread Suyog Rao
Actually, looking at the code, the consumer client code can also catch this exception while iterating for messages. The fetcher thread inserts a special message before dying, which triggers an exception while client calls next message https://github.com/apache/kafka/blob/0.7.2/core/src/main/sca

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-11 Thread vinh
In that case, is there a way to detect that a consumer instance is no longer usable, so that we can recreate the instance on the fly again to have it reconnect? Without having to restart our app? Thanks, -Vinh On Feb 11, 2014, at 7:45 AM, Jun Rao wrote: > We do catch the exception. However,

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-11 Thread Jun Rao
We do catch the exception. However, we don't know what to do with it. Retrying may not fix the problem. So, we just log it and let the thread die. Thanks, Jun On Mon, Feb 10, 2014 at 8:42 PM, Philip O'Toole wrote: > Yes, there might be - we experience link resets every so often, and > definit

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-10 Thread Philip O'Toole
Yes, there might be - we experience link resets every so often, and definitely did today. Assume it is this, are you surprised the thread went down? Perhaps we need to catch this? Philip > On Feb 10, 2014, at 8:38 PM, Jun Rao wrote: > > This indicates that message checksum validation failed

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-10 Thread Jun Rao
This indicates that message checksum validation failed. Is there any issue with the network? Thanks, Jun On Mon, Feb 10, 2014 at 5:00 PM, Philip O'Toole wrote: > Saw this thrown today, which brought down a Consumer thread -- we're using > Consumers built on the High-level consumer framework.

Re: 0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-10 Thread Philip O'Toole
I should we *think* this exception brought down the Consumer thread. The problematic partition on our system was 2-29, so this is definitely the related thread. Philip On Mon, Feb 10, 2014 at 5:00 PM, Philip O'Toole wrote: > Saw this thrown today, which brought down a Consumer thread -- we're

0.72 Consumer: message is invalid, compression codec: NoCompressionCodec

2014-02-10 Thread Philip O'Toole
Saw this thrown today, which brought down a Consumer thread -- we're using Consumers built on the High-level consumer framework. What may have happened here? We are using a custom C++ Producer which does not do compression, and which hasn't changed in months, but this error is relatively new to us,