Cheers!

Nobody has an Idea what to do with InvalidMessageExceptions?

Jörg


-------- Forwarded Message --------
Subject:        MirrorMaker kafka.message.InvalidMessageException
Date:   Fri, 18 Sep 2015 11:37:20 +0200
From:   Jörg Wagner <joerg.wagn...@1und1.de>
Reply-To:       users@kafka.apache.org
To:     users@kafka.apache.org



Hey everyone!

One of my Mirrormakers is exiting with the following error:

[2015-09-18 11:27:35,591] FATAL [mirrormaker-consumer-0] Stream
unexpectedly exited. (kafka.tools.MirrorMaker$ConsumerThread)
kafka.message.InvalidMessageException: Message is corrupt (stored crc =
3256823012, computed crc = 4230971415)
        at kafka.message.Message.ensureValid(Message.scala:166)
        at
kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:101)
        at
kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
        at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
        at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
        at scala.collection.Iterator$class.foreach(Iterator.scala:727)
        at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
        at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25)
        at
kafka.tools.MirrorMaker$ConsumerThread.run(MirrorMaker.scala:259)

The same topic has been mirrored to another DC without a problem.

A few questions arise:

 * Why could the other DC mirror the topic successfully?
 * Why is the MM not failing for the message, but carrying on?
 * What can I do now to carry on consuming with this MM?
 * How can I automate the answers to the above?

My MM currently also does not want to write a log, although I passed on:
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${KAFKA_CONFIG}/mirror/log4j.properties"
Any clues why I don't get any log output? (I got the error from running
MM interactive)

Thanks

Jörg



Reply via email to