nsivabalan commented on code in PR #17928:
URL: https://github.com/apache/hudi/pull/17928#discussion_r3024334609
##########
hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java:
##########
@@ -179,7 +179,31 @@ protected boolean
needUpdatingPersistedRecord(IndexedRecord currentValue,
if (incomingRecord.isEmpty() &&
OrderingValues.isDefault(incomingOrderingVal)) {
return true;
}
- return persistedOrderingVal == null ||
persistedOrderingVal.compareTo(incomingOrderingVal) <= 0;
+ boolean updateOnSameOrderingField =
Boolean.parseBoolean(properties.getProperty(
+ HoodiePayloadProps.UPDATE_ON_SAME_PAYLOAD_ORDERING_FIELD_PROP_KEY,
+
HoodiePayloadProps.DEFAULT_UPDATE_ON_SAME_PAYLOAD_ORDERING_FIELD_PROP_VALUE));
+ return compareOrderingVal(persistedOrderingVal, incomingOrderingVal,
updateOnSameOrderingField);
+ }
+
+ /**
+ * Compares the ordering between persisted entry and input payload.
+ * If updateOnSameOrderingField is true, then incoming record is returned
when payload ordering field is the same.
+ * @param persistedOrderingVal record present in Disk
+ * @param incomingOrderingVal record part of input payload
+ * @return true if the older record(persisted entry) is older than incoming
record.
+ */
+ protected boolean compareOrderingVal(Comparable persistedOrderingVal,
Comparable incomingOrderingVal,
Review Comment:
hey @danny0405 : Uber is upstreaming all of their internal patches which
they have been using it in their internal fork. Their plan is to use hudi OSS
1.2 to upgrade and not have any internal changes as much as possible.
Think of it like a tooling that they are upstreaming. in OSS as such, we are
not going to call this out to anyone (in release docs or anywhere).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
