yihua commented on code in PR #13642:
URL: https://github.com/apache/hudi/pull/13642#discussion_r2257631292
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -275,4 +303,42 @@ protected Pair<Map<ConfigProperty, String>,
List<ConfigProperty>> downgrade(Hood
throw new HoodieUpgradeDowngradeException(fromVersion.versionCode(),
toVersion.versionCode(), false);
}
}
+
+ /**
+ * Checks if any handlers in the upgrade/downgrade path require running
rollback and compaction before starting process.
+ *
+ * @param fromVersion the current table version
+ * @param toVersion the target table version
+ */
+ private void performRollbackAndCompactionIfRequired(HoodieTableVersion
fromVersion, HoodieTableVersion toVersion, boolean isUpgrade) {
Review Comment:
Let's separate the discussion on rollback and compaction.
Rollback always happens during the upgrade process in 0.x releases (see
`BaseHoodieWriteClient#tryUpgrade`) so the rollback inside `rollbackAndCompact`
is a no-op if the upgrade is invoked from the write client. We should rewrite
the logic to avoid duplicate code.
Compaction should only happen during 6->8 upgrade because of the changes in
log file name and completion time semantics and conditionally during 8->9
upgrade for certain payloads because of payload deprecation (assuming this is
still needed). Compaction should not always happen since it's a IO and compute
intensive procedure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
