yihua opened a new issue, #19999:
URL: https://github.com/apache/hudi/issues/19999
`HoodieFileGroupReader` can hand back the log records of a file slice only
as a stream, via `getLogRecordsOnly()`. A caller that needs to look a record up
by key, or by position, has to drain that iterator and rebuild a map, which
duplicates in memory what the record buffer is already holding.
The older reader does not have this gap. `HoodieMergedLogRecordReader`
exposes both halves over the same buffer:
@Override
public Iterator<BufferedRecord<T>> iterator() {
return recordBuffer.getLogRecordIterator();
}
public Map<Serializable, BufferedRecord<T>> getRecords() {
return recordBuffer.getLogRecords();
}
`HoodieFileGroupReader` has only the iterator half, so code moving from the
merged log record reader to the file group reader loses random access.
Proposal: add the matching accessor to `HoodieFileGroupReader`, returning
the buffer's map keyed by record key, or by record position when the reader is
merging on positions, the same keying the buffer already uses internally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
