nsivabalan commented on code in PR #13642:
URL: https://github.com/apache/hudi/pull/13642#discussion_r2244197075
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -275,4 +307,44 @@ protected Pair<Map<ConfigProperty, String>,
List<ConfigProperty>> downgrade(Hood
throw new HoodieUpgradeDowngradeException(fromVersion.versionCode(),
toVersion.versionCode(), false);
}
}
+
+ /**
+ * Checks if any handlers in the upgrade/downgrade path need compaction and
performs it once before starting.
+ * This ensures compaction happens only when needed at the very beginning of
the process.
+ *
+ * @param fromVersion the current table version
+ * @param toVersion the target table version
+ */
+ private boolean requiresCompaction(HoodieTableVersion fromVersion,
HoodieTableVersion toVersion) {
Review Comment:
why can't we pass the `isUpgrade` boolean from above as an argument here and
avoid L325
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -130,6 +145,29 @@ public boolean needsUpgrade(HoodieTableVersion
toWriteVersion) {
* @param instantTime current instant time that should not be touched.
*/
public void run(HoodieTableVersion toVersion, String instantTime) {
+ // Fetch version from property file and current version
+ HoodieTableVersion fromVersion =
metaClient.getTableConfig().getTableVersion();
+ // Determine if we are upgrading or downgrading
+ boolean isUpgrade = fromVersion.versionCode() < toVersion.versionCode();
+ if (isUpgrade && !config.autoUpgrade()) {
+ // if we are attempting to upgrade and auto-upgrade is disabled
+ // we set the write config table version to bounded by the current hudi
table version
+ // and then exit out the upgrade process
+ config.setValue(HoodieWriteConfig.WRITE_TABLE_VERSION,
String.valueOf(fromVersion));
Review Comment:
if auto upgrade is disabled, we don't need to set anything here.
just log a warn msg and return right away.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -144,16 +154,12 @@ public void run(HoodieTableVersion toVersion, String
instantTime) {
.run(toVersion, instantTime);
}
} catch (Exception e) {
- LOG.warn("Unable to upgrade or downgrade the metadata table to version
" + toVersion
- + ", ignoring the error and continue.", e);
+ throw new HoodieUpgradeDowngradeException("Upgrade/downgrade for the
Hudi metadata table failed. "
Review Comment:
ok. did you get past the issue?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -150,6 +150,7 @@ public class HoodieWriteConfig extends HoodieConfig {
.defaultValue(HoodieTableVersion.current().versionCode())
.withValidValues(
String.valueOf(HoodieTableVersion.SIX.versionCode()),
+ String.valueOf(HoodieTableVersion.SEVEN.versionCode()),
Review Comment:
sg
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
