Cheers!
Nobody has an Idea what to do with InvalidMessageExceptions?
Jörg
-------- Forwarded Message --------
Subject: MirrorMaker kafka.message.InvalidMessageException
Date: Fri, 18 Sep 2015 11:37:20 +0200
From: Jörg Wagner <joerg.wagn...@1und1.de>
Reply-To: users@kafka.apache.org
To: users@kafka.apache.org
Hey everyone!
One of my Mirrormakers is exiting with the following error:
[2015-09-18 11:27:35,591] FATAL [mirrormaker-consumer-0] Stream
unexpectedly exited. (kafka.tools.MirrorMaker$ConsumerThread)
kafka.message.InvalidMessageException: Message is corrupt (stored crc =
3256823012, computed crc = 4230971415)
at kafka.message.Message.ensureValid(Message.scala:166)
at
kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:101)
at
kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25)
at
kafka.tools.MirrorMaker$ConsumerThread.run(MirrorMaker.scala:259)
The same topic has been mirrored to another DC without a problem.
A few questions arise:
* Why could the other DC mirror the topic successfully?
* Why is the MM not failing for the message, but carrying on?
* What can I do now to carry on consuming with this MM?
* How can I automate the answers to the above?
My MM currently also does not want to write a log, although I passed on:
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${KAFKA_CONFIG}/mirror/log4j.properties"
Any clues why I don't get any log output? (I got the error from running
MM interactive)
Thanks
Jörg
