apoorvmittal10 opened a new pull request, #20395:
URL: https://github.com/apache/kafka/pull/20395
The PR fixes the batch alignment issue when partitions are re-assigned.
During initial read of state the batches can be broken arbitrarily. Say the
start offset is 10 and cache contains [15-18] batch during initialization. When
fetch happens at offset 10 and say the fetched batch contain 10 records i.e.
[10-19] then correct batches will be created if maxFetchRecords is greater than
10. But if maxFetchRecords is less than 10 then last offset of batch is
determined, which will be 19. Hence acquire method will incorrectly create a
batch of [10-19] while [15-18] already exists. Below check is required t
resolve the issue:
```
if (isInitialReadGapOffsetWindowActive() && lastAcquiredOffset > lastOffset)
{
lastAcquiredOffset = lastOffset;
}
```
While testing with other cases, other issues were determined while updating
the gap offset, acquire of records prior share partitions end offset and
determining next fetch offset with compacted topics. All these issues can arise
mainly during initial read window after partition re-assignment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
