Cheers! Nobody has an Idea what to do with InvalidMessageExceptions?
Jörg -------- Forwarded Message -------- Subject: MirrorMaker kafka.message.InvalidMessageException Date: Fri, 18 Sep 2015 11:37:20 +0200 From: Jörg Wagner <joerg.wagn...@1und1.de> Reply-To: users@kafka.apache.org To: users@kafka.apache.org Hey everyone! One of my Mirrormakers is exiting with the following error: [2015-09-18 11:27:35,591] FATAL [mirrormaker-consumer-0] Stream unexpectedly exited. (kafka.tools.MirrorMaker$ConsumerThread) kafka.message.InvalidMessageException: Message is corrupt (stored crc = 3256823012, computed crc = 4230971415) at kafka.message.Message.ensureValid(Message.scala:166) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:101) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25) at kafka.tools.MirrorMaker$ConsumerThread.run(MirrorMaker.scala:259) The same topic has been mirrored to another DC without a problem. A few questions arise: * Why could the other DC mirror the topic successfully? * Why is the MM not failing for the message, but carrying on? * What can I do now to carry on consuming with this MM? * How can I automate the answers to the above? My MM currently also does not want to write a log, although I passed on: KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${KAFKA_CONFIG}/mirror/log4j.properties" Any clues why I don't get any log output? (I got the error from running MM interactive) Thanks Jörg