stayrascal commented on a change in pull request #4724:
URL: https://github.com/apache/hudi/pull/4724#discussion_r813849545
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java
##########
@@ -58,6 +58,31 @@ default T preCombine(T oldValue, Properties properties) {
return preCombine(oldValue);
}
+ /**
+ *When more than one HoodieRecord have the same HoodieKey in the incoming
batch, this function combines them before attempting to insert/upsert by taking
in a property map.
+ *
+ * @param oldValue instance of the old {@link HoodieRecordPayload} to be
combined with.
+ * @param properties Payload related properties. For example pass the
ordering field(s) name to extract from value in storage.
+ * @param schema Schema used for record
+ * @return the combined value
+ */
+ @PublicAPIMethod(maturity = ApiMaturityLevel.STABLE)
+ default T preCombine(T oldValue, Properties properties, Schema schema) {
Review comment:
Thanks @alexeykudinkin for reviewing this.
What we are trying to do is implement partial update purpose. For example,
let's assume the record schema is `(f0 int , f1 int, f2 int)`, The first record
value is: (1, 2, 3), the second record value is: (4, 5, null) with the field f2
value as null. We hope that the result after run `preCombine` is `(4, 5, 3)`,
which means we need to combine/merge two records to a third one, not only
choose one of them.
Actually, what we want to implement is similar with
`#combineAndGetUpdateValue(IndexedRecord currentValue, Schema schema)` which
used for combine the incoming record with existing record from base/log file.
But `#preCombine` will be used for combing/merging two incoming records in a
batch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
