[
https://issues.apache.org/jira/browse/KAFKA-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-12619:
------------------------------------
Description:
KIP-595 describes an extra condition on commitment here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-Fetch.
In order to ensure that a newly elected leader's committed entries cannot get
lost, it must commit one record from its own epoch. This guarantees that its
latest entry is larger (in terms of epoch/offset) than any previously written
record which ensures that any future leader must also include it. This is the
purpose of the LeaderChange record which is written to the log as soon as the
leader gets elected.
We have this check implemented here:
https://github.com/apache/kafka/blob/trunk/raft/src/main/java/org/apache/kafka/raft/LeaderState.java#L122.
However, the check needs to be a strict inequality since the epoch start
offset does not reflect the LeaderChange record itself. In other words, the
check is off by one.
was:
KIP-595 describes an extra condition on commitment here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-Fetch.
In order to ensure that the leader's committed entries cannot get lost, it
must commit one record from its own epoch. This guarantees that its latest
entry is larger (in terms of epoch/offset) than any previously written record
which ensures that any future leader must also include it. This is the purpose
of the LeaderChange record which is written to the log as soon as the leader
gets elected.
We have this check implemented here:
https://github.com/apache/kafka/blob/trunk/raft/src/main/java/org/apache/kafka/raft/LeaderState.java#L122.
However, the check needs to be a strict inequality since the epoch start
offset does not reflect the LeaderChange record itself. In other words, the
check is off by one.
> Ensure LeaderChange message is committed before initializing high watermark
> ---------------------------------------------------------------------------
>
> Key: KAFKA-12619
> URL: https://issues.apache.org/jira/browse/KAFKA-12619
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Major
>
> KIP-595 describes an extra condition on commitment here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-Fetch.
> In order to ensure that a newly elected leader's committed entries cannot
> get lost, it must commit one record from its own epoch. This guarantees that
> its latest entry is larger (in terms of epoch/offset) than any previously
> written record which ensures that any future leader must also include it.
> This is the purpose of the LeaderChange record which is written to the log as
> soon as the leader gets elected.
> We have this check implemented here:
> https://github.com/apache/kafka/blob/trunk/raft/src/main/java/org/apache/kafka/raft/LeaderState.java#L122.
> However, the check needs to be a strict inequality since the epoch start
> offset does not reflect the LeaderChange record itself. In other words, the
> check is off by one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
