nsivabalan commented on code in PR #13830:
URL: https://github.com/apache/hudi/pull/13830#discussion_r2319928557
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java:
##########
@@ -638,7 +637,7 @@ public static <R> HoodieData<HoodieRecord<R>>
tagGlobalLocationBackToRecords(
}
});
return shouldUpdatePartitionPath
- ? mergeForPartitionUpdatesIfNeeded(incomingRecordsAndLocations,
config, table)
+ ? mergeForPartitionUpdatesIfNeeded(incomingRecordsAndLocations,
config, table, readerContext, writerSchema)
Review Comment:
if `shouldUpdatePartitionPath` is false (bcoz
hoodie.record.index.update.partition.path=false is set), then wouldn't the
incoming record will be routed to previous partition path's file group.
and eventually HoodieMergeHandle or MOR snapshot read will take care of
reconciling right?
here mainly, we are tackling scenarios where records move from one partition
to other, we need to merge the incoming record w/ previous version of the
record which could be in a diff file group and hence write handles may not be
able to handle them as it might go into diff write handles.
I agree to your 2nd point. we can make
`hoodie.record.index.update.partition.path` default to true. I don't really
remember why we kept it false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
