cshuo commented on code in PR #13498:
URL: https://github.com/apache/hudi/pull/13498#discussion_r2191345314
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/BufferedRecordMergerFactory.java:
##########
@@ -98,12 +122,28 @@ public Pair<Boolean, T> finalMerge(BufferedRecord<T>
olderRecord, BufferedRecord
* based on {@code EVENT_TIME_ORDERING} merge mode.
*/
private static class EventTimeBufferedRecordMerger<T> implements
BufferedRecordMerger<T> {
+ private final PartialUpdateStrategy<T> partialUpdateStrategy;
+ private final Schema readerSchema;
+
+ public EventTimeBufferedRecordMerger(HoodieReaderContext<T> readerContext,
+ PartialUpdateMode partialUpdateMode,
+ TypedProperties props,
+ Schema readerSchema) {
+ this.partialUpdateStrategy = new PartialUpdateStrategy<>(readerContext,
partialUpdateMode, props);
+ this.readerSchema = readerSchema;
+ }
+
@Override
public Option<BufferedRecord<T>> deltaMerge(BufferedRecord<T> newRecord,
BufferedRecord<T> existingRecord) {
if (existingRecord == null || shouldKeepNewerRecord(existingRecord,
newRecord)) {
+ newRecord = partialUpdateStrategy.reconcileFieldsWithOldRecord(
+ newRecord, existingRecord, readerSchema, readerSchema, false);
return Option.of(newRecord);
+ } else {
+ existingRecord = partialUpdateStrategy.reconcileFieldsWithOldRecord(
Review Comment:
Here may introduce performance regression. It
seems`reconcileFieldsWithOldRecord` never return null, so now `deltaMerge`
never return empty, while previously we return empty option to skip updating
the record buffer. We should align the behavior when partial update mode is
None.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
