[
https://issues.apache.org/jira/browse/KAFKA-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101993#comment-15101993
]
Flavio Junqueira commented on KAFKA-3083:
-----------------------------------------
Hey [~mgharat], Best practice with ZK is to put the master (controller in this
case) on hold upon a connection loss event and wait until the next event, which
can be flagging a reconnection or that the session has expired. It should call
{{controllerResignation}} upon a session expiration, and resume if it
reconnects.
But, we have to be careful because we can't really control the speed of
messages, and even if A stops before B takes over in your example, we can't
guarantee that some message from A will hit some broker somewhere late. The
description of this jira says that C correctly discards an old message, and it
should be like that, so this part looks fine this far. It is about the change
in ZK happening at the wrong time.
> a soft failure in controller may leader a topic partition in an inconsistent
> state
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-3083
> URL: https://issues.apache.org/jira/browse/KAFKA-3083
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.9.0.0
> Reporter: Jun Rao
> Assignee: Mayuresh Gharat
>
> The following sequence can happen.
> 1. Broker A is the controller and is in the middle of processing a broker
> change event. As part of this process, let's say it's about to shrink the isr
> of a partition.
> 2. Then broker A's session expires and broker B takes over as the new
> controller. Broker B sends the initial leaderAndIsr request to all brokers.
> 3. Broker A continues by shrinking the isr of the partition in ZK and sends
> the new leaderAndIsr request to the broker (say C) that leads the partition.
> Broker C will reject this leaderAndIsr since the request comes from a
> controller with an older epoch. Now we could be in a situation that Broker C
> thinks the isr has all replicas, but the isr stored in ZK is different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
