yihua opened a new issue, #20000: URL: https://github.com/apache/hudi/issues/20000
`HoodieFileGroupReader#initRecordIterators()` unconditionally reassigns `baseFileIterator` and `recordBuffer`, and `close()` only closes whichever pair is current. Every accessor that needs the reader initialized calls it: `getClosableIterator()`, `getClosableBufferedRecordIterator()`, `getClosableHoodieRecordIterator()`, `getClosableKeyIterator()` and `getLogRecordsOnly()`. So two accessor calls on one reader build two record buffers, and the first is never closed. Its `ExternalSpillableMap` holds an open disk map when the read has spilled, which then survives the reader. The second call also re-opens the base file and re-scans every log file in the slice, so the cost is not only the leak. A reader is usually consumed once, which is why this has not bitten, but the class does not document or enforce that, and nothing about the accessors suggests they are mutually exclusive. The obvious fix, returning early when already initialized, is not safe on its own: `getBufferedRecordIterator(IteratorMode)` sets the iterator mode on the reader context *before* calling `initRecordIterators()`, so sharing one buffer across calls would serve a buffer built under a different mode. A fix needs to either close the previous buffer and base iterator before rebuilding, or make the iterator mode part of what the reader is built with rather than something set per accessor call. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
