[
https://issues.apache.org/jira/browse/KAFKA-17801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamal Chandraprakash reopened KAFKA-17801:
------------------------------------------
> RemoteLogManager may compute inaccurate upperBoundOffset for aborted txns
> -------------------------------------------------------------------------
>
> Key: KAFKA-17801
> URL: https://issues.apache.org/jira/browse/KAFKA-17801
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 3.6.0
> Reporter: Jun Rao
> Assignee: Kamal Chandraprakash
> Priority: Major
> Fix For: 4.0.0
>
>
> In RemoteLogManager.read, we compute startPos as the following.
> {code:java}
> startPos = lookupPositionForOffset(remoteLogSegmentMetadata, offset);{code}
> This is the position returned by the offset index. The actual position for
> the first batch being read happens in the following, but startPos is not
> updated accordingly.
> {code:java}
> firstBatch = findFirstBatch(remoteLogInputStream, offset);{code}
> We then use the inaccurate startPos to create fetchDataInfo.
> {code:java}
> FetchDataInfo fetchDataInfo = new FetchDataInfo(
> new LogOffsetMetadata(offset, remoteLogSegmentMetadata.startOffset(),
> startPos),
> MemoryRecords.readableRecords(buffer));{code}
> In addAbortedTransactions(), we use startPos to find the upperBoundOffset to
> retrieve the aborted txns.
> {code:java}
> long upperBoundOffset =
> offsetIndex.fetchUpperBoundOffset(startOffsetPosition, fetchSize)
> .map(position -> position.offset).orElse(segmentMetadata.endOffset() +
> 1);{code}
> The inaccurate startPos can lead to inaccurate upperBoundOffset, which leads
> to inaccurate aborted txns returned to the consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
