[
https://issues.apache.org/jira/browse/KAFKA-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997087#comment-14997087
]
Jiangjie Qin commented on KAFKA-2747:
-------------------------------------
[~geoffra] Not sure how this happens. I dumped the log segments from source and
target cluster, found the following messages are missing:
offset 34291 (target) ----> offset 34079 (source)
offset 34292 (target) ----> offset 57421 (source)
{noformat}
>From mirror maker log it does not seems have any problem.
[2015-11-09 00:18:43,870] INFO Starting mirror maker (kafka.tools.MirrorMaker$)
....
[2015-11-09 00:19:17,744] DEBUG Committed offset 28549 for partition topic-0
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
..... (hard kill)
[2015-11-09 00:19:23,772] INFO Starting mirror maker (kafka.tools.MirrorMaker$)
....
[2015-11-09 00:19:17,744] DEBUG Committed offset 28549 for partition topic-0
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
...
[2015-11-09 00:19:49,099] DEBUG Committed offset 57421 for partition topic-0
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
... (hard kill)
[2015-11-09 00:19:55,351] INFO Starting mirror maker (kafka.tools.MirrorMaker$)
....
[2015-11-09 00:20:16,608] DEBUG Resetting offset for partition topic-0 to the
committed offset 57421 (org.apache.kafka.clients.consumer.internals.Fetcher)
....
[2015-11-09 00:20:19,076] DEBUG Committed offset 87423 for partition topic-0
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
....(Another hard kill)
[2015-11-09 00:20:26,268] INFO Starting mirror maker (kafka.tools.MirrorMaker$)
....
[2015-11-09 00:20:46,612] DEBUG Resetting offset for partition topic-0 to the
committed offset 87423 (org.apache.kafka.clients.consumer.internals.Fetcher)
...
[2015-11-09 00:21:52,803] DEBUG Committed offset 120567 for partition topic-0
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
...
[2015-11-09 00:21:52,829] INFO [mirrormaker-thread-0] Mirror maker thread
shutdown complete (kafka.tools.MirrorMaker$MirrorMakerThread)
{noformat}
It looks mirror maker committed offset 57421 when the messages between 34079
and 57421 haven't been sent successfully. I am not sure how this happened
because in mirror maker we always call producer.flush() before committing
offsets. Can you turn on trace level logging on MirrorMaker and broker for
further debugging?
> Message loss if mirror maker is killed with hard kill and then restarted
> ------------------------------------------------------------------------
>
> Key: KAFKA-2747
> URL: https://issues.apache.org/jira/browse/KAFKA-2747
> Project: Kafka
> Issue Type: Bug
> Reporter: Geoff Anderson
> Attachments: 2015-11-08--004-dataloss-newconsumer.tar.gz
>
>
> I recently added simple failover to the existing mirror maker test
> (https://github.com/apache/kafka/pull/427) and found that killing mirror
> maker process with a hard kill resulted in message loss.
> The test here has two single-node broker clusters, one producer producing to
> the source cluster, one consumer consuming from the target cluster, and a
> single mirror maker instance mirroring data between the two clusters.
> mirror maker is using old consumer, zookeeper for offset storage
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
