[
https://issues.apache.org/jira/browse/KAFKA-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584955#comment-13584955
]
Jun Rao commented on KAFKA-763:
-------------------------------
The intention of this jira is just to avoid fetching too much data than
necessary, where the follower hits OffsetOutOfRangeException. There are 2
common cases when this can happen.
1. There is an unclean leader election and the follower's offset is larger than
leader's last offset. In this case, the follower should truncate its log to the
leader's last offset and fetch from that offset.
2. A follower has been down for a long time and when it starts up, its offset
is smaller than the smallest offset in the leader (because old logs have been
deleted). In this case, the follower should truncate its log to the leader's
first offset and fetch from that offset.
So, when a follower receives an OffsetOutOfRangeException, it probably should
do the following:
1. get the last offset LO from leader
2. if LO is within the offset range of the follower, follower truncates its log
to LO and fetch from LO.
3. otherwise, follower gets the first offset FO from leader, deletes all data
in its log and starts with an empty log with offset FO, fetches from FO.
There a separate issue that when there is an unclean leader election, after
follower recovers, the replicas may not be identical for a chunk of offsets.
It's possible, but not easy to fix this issue. We can follow up on this one
post 0.8.
> Add an option to replica from the largest offset during unclean leader
> election
> -------------------------------------------------------------------------------
>
> Key: KAFKA-763
> URL: https://issues.apache.org/jira/browse/KAFKA-763
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: p2
> Attachments: kafka-763_v1.patch
>
>
> If there is an unclean leader election, a follower may have an offset out of
> the range of the leader. Currently, the follower will delete all its data and
> refetch from the smallest offset of the leader. It would be useful to add an
> option to let the follower refetch from the largest offset of the leader
> since refetching from the smallest offset may take some time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
