[
https://issues.apache.org/jira/browse/KAFKA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616151#comment-15616151
]
TUSHAR SHARAD MHASKAR edited comment on KAFKA-3772 at 10/28/16 6:20 PM:
------------------------------------------------------------------------
I have seen this same issue with 0.9.0.0 Mirror Maker version as well.
was (Author: tushr1388):
I have seen this same issue with 0.9.0.0 version as well.
> MirrorMaker crashes on Corrupted Message
> ----------------------------------------
>
> Key: KAFKA-3772
> URL: https://issues.apache.org/jira/browse/KAFKA-3772
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.9.0.1
> Reporter: James Ranson
> Labels: mirror-maker
>
> We recently came across an issue where a message on our source kafka cluster
> became corrupted. When MirrorMaker tried to consume this message, the thread
> crashed and caused the entire process to also crash. Each time we attempted
> to restart MM, it crashed on the same message. There is no information in the
> MM logs about which message it was trying to consume (what topic, what
> offset, etc). So the only way we were able to get past the issue was to go
> into the zookeeper tree for our mirror consumer group and increment the
> offset for every partition on every topic until the MM process could start
> without crashing. This is not a tenable operational solution. MirrorMaker
> should gracefully skip corrupt messages since they will never be able to be
> replicated anyway.
> {noformat}2016-05-26 20:02:26,787 FATAL MirrorMaker$MirrorMakerThread - [{}]
> [mirrormaker-thread-3] Mirror maker thread failure due to
> kafka.message.InvalidMessageException: Message is corrupt (stored crc =
> 33747148, computed crc = 3550736267)
> at kafka.message.Message.ensureValid(Message.scala:167)
> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:101)
> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> at
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> at
> kafka.tools.MirrorMaker$MirrorMakerOldConsumer.hasData(MirrorMaker.scala:483)
> at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:394)
> 2016-05-26 20:02:27,580 FATAL MirrorMaker$MirrorMakerThread - [{}]
> [mirrormaker-thread-3] Mirror maker thread exited abnormally, stopping the
> whole mirror maker.{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
