[
https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881262#comment-13881262
]
Andrew Olson commented on KAFKA-1028:
-------------------------------------
Yes, it is a more robust solution than simply requiring 2 acks or a majority
quorum. If the ISR ever becomes empty, there is a significant risk that the
next leader will not be the same as the previous one, likely resulting in data
inconsistency.
Consider the following example sequence of events, given a cluster with 4
brokers, a replication factor of 4, and 2 required acks.
1. Brokers A and B die simultaneously and are removed from the initially
complete ISR set of (A,B,C,D). If A and B were leaders for any partitions,
leadership is transferred to C or D. Messages can still be received and
consumed by C and D, since ISR = (C,D).
2. The other two surviving brokers C and D are both then shutdown. The entire
cluster is offline, so ISR = ().
3. Next, brokers A and B are restarted. As the only available brokers, they are
the only candidates to be elected leader for all partitions, and ISR now =
(A,B). This is called an "unclean" election since A and B were previously
evicted from the ISR. All messages replicated to A and B before their sudden
deaths are available for consumption, and new messages can also be received.
4. Finally, brokers C and D are restarted. Since A and B are now the leaders
for everything, C and D must assume the role of follower replicas. When
synchronizing the leader and follower logs, the messages which were received by
C and D while they were still alive when A and B were dead can be lost by log
file truncation if A or B now has a lower latest offset value. There can also
potentially be different message data in the logs for the same offset values.
In this scenario, this enhancement would introduce the configurable option to
prevent the unclean leader election of A and B on step 3 above, so that the
data remains consistent. However, the tradeoff to ensuring consistency is that
the topics in this cluster would be unavailable to produce to or consume from
until either C or D is revived.
> per topic configuration of preference for consistency over availability
> -----------------------------------------------------------------------
>
> Key: KAFKA-1028
> URL: https://issues.apache.org/jira/browse/KAFKA-1028
> Project: Kafka
> Issue Type: Improvement
> Reporter: Scott Clasen
> Assignee: Neha Narkhede
>
> As discussed with Neha on the ML.
> It should be possible to configure a topic to disallow unclean leader
> election, thus preventing the situation where committed messages can be
> discarded once a failed leader comes back online in a situation where it was
> the only ISR.
> This would open kafka to additional usecases where the possibility of
> committted messages being discarded is unacceptable.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
