nsivabalan commented on code in PR #13229:
URL: https://github.com/apache/hudi/pull/13229#discussion_r2079039337
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieTableServiceClient.java:
##########
@@ -369,6 +379,23 @@ protected void completeCompaction(HoodieCommitMetadata
metadata, HoodieTable tab
LOG.info("Compacted successfully on commit {}", compactionCommitTime);
}
+ public void commitLogCompaction(String compactionInstantTime,
HoodieWriteMetadata<O> writeMetadata, Option<HoodieTable> tableOpt) {
+ // dereferencing the write dag for log compaction for the first time.
+ List<HoodieWriteStat> writeStats =
triggerWritesAndFetchWriteStats(writeMetadata);
Review Comment:
https://github.com/apache/hudi/pull/13236 -> we are going to need this
abstraction in this follow up patch.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieTableServiceClient.java:
##########
@@ -481,23 +508,15 @@ public HoodieWriteMetadata<O> cluster(String
clusteringInstant, boolean shouldCo
throw new HoodieClusteringException("Non clustering replace-commit
inflight at timestamp " + clusteringInstant);
}
}
+
clusteringTimer = metrics.getClusteringCtx();
LOG.info("Starting clustering at {} for table {}", clusteringInstant,
table.getConfig().getBasePath());
HoodieWriteMetadata<T> writeMetadata = table.cluster(context,
clusteringInstant);
HoodieWriteMetadata<O> clusteringMetadata =
convertToOutputMetadata(writeMetadata);
- // Publish file creation metrics for clustering.
- if (config.isMetricsOn()) {
- clusteringMetadata.getWriteStats()
- .ifPresent(hoodieWriteStats -> hoodieWriteStats.stream()
- .filter(hoodieWriteStat -> hoodieWriteStat.getRuntimeStats() !=
null)
- .map(hoodieWriteStat ->
hoodieWriteStat.getRuntimeStats().getTotalCreateTime())
- .forEach(metrics::updateClusteringFileCreationMetrics));
- }
-
// TODO : Where is shouldComplete used ?
- if (shouldComplete && clusteringMetadata.getCommitMetadata().isPresent()) {
- completeClustering((HoodieReplaceCommitMetadata)
clusteringMetadata.getCommitMetadata().get(), table, clusteringInstant);
+ if (shouldComplete) {
+ commitClustering(clusteringMetadata, table, clusteringInstant);
Review Comment:
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
