danny0405 commented on code in PR #13292:
URL: https://github.com/apache/hudi/pull/13292#discussion_r2113028448
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2855,16 +2864,25 @@ public Integer getWritesFileIdEncoding() {
return props.getInteger(WRITES_FILEID_ENCODING,
HoodieMetadataPayload.RECORD_INDEX_FIELD_FILEID_ENCODING_UUID);
}
- public boolean needResolveWriteConflict(WriteOperationType operationType) {
+ public boolean needResolveWriteConflict(WriteOperationType operationType,
boolean isMetadataTable, HoodieWriteConfig config,
+ HoodieTableConfig tableConfig) {
WriteConcurrencyMode mode = getWriteConcurrencyMode();
switch (mode) {
case SINGLE_WRITER:
return false;
case OPTIMISTIC_CONCURRENCY_CONTROL:
return true;
- case NON_BLOCKING_CONCURRENCY_CONTROL:
- // NB-CC don't need to resolve write conflict except bulk insert
operation
- return WriteOperationType.BULK_INSERT == operationType;
+ case NON_BLOCKING_CONCURRENCY_CONTROL: {
+ if (isMetadataTable &&
config.isStreamingWritesToMetadataEnabled(tableConfig.getTableVersion())) {
+ // datatable NBCC is still evolving and might go through evolution
compared to its current state.
+ // But incase of metadata table, when streaming writes are enabled,
we are leveraging MetadataTableNonBlockingWritesConflictResolutionStrategy to
do conflict resolution
+ // still keeping future capabilities open to enhance
MetadataTableNonBlockingWritesConflictResolutionStrategy.
Review Comment:
It looks like we already have `SINGLE_WRITER` concurrency mode to skip
conflict resolution for MDT on master, this might be addidional overhead
introduced:
```java
// BaseHoodieWriteClient.preWrite
this.lastCompletedTxnAndMetadata = txnManager.isLockRequired()
? TransactionUtils.getLastCompletedTxnInstantAndMetadata(metaClient)
: Option.empty();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
