yihua commented on code in PR #13670:
URL: https://github.com/apache/hudi/pull/13670#discussion_r2250239631
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -128,14 +128,14 @@ private HoodieFileGroupReader(HoodieReaderContext<T>
readerContext, HoodieStorag
*/
private void initRecordIterators() throws IOException {
ClosableIterator<T> iter = makeBaseFileIterator();
- if (inputSplit.getLogFiles().isEmpty()) {
+ Option<Pair<HoodieFileGroupRecordBuffer<T>, List<String>>>
initializationResultOpt = recordBufferLoader.getRecordBuffer(
+ readerContext, storage, inputSplit, orderingFieldNames, metaClient,
props, readerParameters, readStats, fileGroupUpdateCallback);
Review Comment:
I assume the intention is that the new "updates" can either come from the
log files (from storage for read, compaction, etc.) or record iterator (from
ingestion input for the merge handle). Is there a plan to provide a new
abstraction to represent this instead of `InputSplit` which currently
represents base and log files only?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -128,14 +128,14 @@ private HoodieFileGroupReader(HoodieReaderContext<T>
readerContext, HoodieStorag
*/
private void initRecordIterators() throws IOException {
ClosableIterator<T> iter = makeBaseFileIterator();
- if (inputSplit.getLogFiles().isEmpty()) {
+ Option<Pair<HoodieFileGroupRecordBuffer<T>, List<String>>>
initializationResultOpt = recordBufferLoader.getRecordBuffer(
+ readerContext, storage, inputSplit, orderingFieldNames, metaClient,
props, readerParameters, readStats, fileGroupUpdateCallback);
Review Comment:
This can be done separate, to make the code easier to follow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
