José Armando García Sancio created KAFKA-15021:
--------------------------------------------------
Summary: KRaft controller increases leader epoch when shrinking ISR
Key: KAFKA-15021
URL: https://issues.apache.org/jira/browse/KAFKA-15021
Project: Kafka
Issue Type: Bug
Components: controller, kraft
Reporter: José Armando García Sancio
Assignee: José Armando García Sancio
When the KRaft controller shrinks the ISR it also forces the leader epoch to
increase. This is unnecessary and cases all of the follower replica fetches to
get invalidated.
Here is an example trace of this behavior after replica 8 was shutdown:
{code:java}
kafka-dump-log --cluster-metadata-decoder --files
__cluster_metadata-0/00000000000038589501.log | grep Pd7wMb4lSkKI00--SrWNXw
...
| offset: 38655592 CreateTime: 1683849857362 keySize: -1 valueSize: 41
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":7,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[3,1],"leader":1}}
| offset: 38655593 CreateTime: 1683849857362 keySize: -1 valueSize: 41
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":5,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[0,4],"leader":4}}
| offset: 38655594 CreateTime: 1683849857362 keySize: -1 valueSize: 41
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":6,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[0,1],"leader":0}}
| offset: 38656159 CreateTime: 1683849974945 keySize: -1 valueSize: 39
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":7,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[3,1,8]}}
| offset: 38656256 CreateTime: 1683849994297 keySize: -1 valueSize: 39
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":5,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[0,4,8]}}
| offset: 38656299 CreateTime: 1683849997139 keySize: -1 valueSize: 39
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":6,"topicId":"Pd7wMb4lSkKI00--SrWNXw","isr":[0,1,8]}}
| offset: 38657003 CreateTime: 1683850157379 keySize: -1 valueSize: 30
sequence: -1 headerKeys: [] payload:
{"type":"PARTITION_CHANGE_RECORD","version":0,"data":{"partitionId":7,"topicId":"Pd7wMb4lSkKI00--SrWNXw","leader":8}}
{code}
Also, notice how the leader epoch was not increased when the ISR was expanded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
