yihua commented on code in PR #13687:
URL: https://github.com/apache/hudi/pull/13687#discussion_r2267359243
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -85,19 +85,44 @@ public UpgradeDowngrade(
public static boolean needsUpgradeOrDowngrade(HoodieTableMetaClient
metaClient, HoodieWriteConfig config, HoodieTableVersion toWriteVersion) {
HoodieTableVersion fromTableVersion =
metaClient.getTableConfig().getTableVersion();
- return needsUpgrade(metaClient, config, toWriteVersion) ||
toWriteVersion.versionCode() < fromTableVersion.versionCode();
+ return needsUpgrade(metaClient, config, toWriteVersion) ||
needsDowngrade(fromTableVersion, toWriteVersion);
+ }
+
+ public static boolean needsDowngrade(HoodieTableVersion fromTableVersion,
HoodieTableVersion toWriteVersion) {
+ if (toWriteVersion.lesserThan(HoodieTableVersion.SIX)) {
+ // for 1.1 we will do not allow downgrades to below SIX
+ // user will have to downgrade the table using a prior hudi version.
+ throw new HoodieUpgradeDowngradeException(
+ String.format("1.1.0 only supports table version greater then
version SIX or above."
+ + " Please downgrade table from version %s to %s using a
hudi version prior to 1.1.0", fromTableVersion, toWriteVersion));
Review Comment:
nit: typo
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -90,14 +90,34 @@ public static boolean
needsUpgradeOrDowngrade(HoodieTableMetaClient metaClient,
public static boolean needsUpgrade(HoodieTableMetaClient metaClient,
HoodieWriteConfig config, HoodieTableVersion toWriteVersion) {
HoodieTableVersion fromTableVersion =
metaClient.getTableConfig().getTableVersion();
- // If table version is less than SIX, then we need to upgrade to SIX first
before upgrading to any other version, irrespective of autoUpgrade flag
+ if (fromTableVersion.versionCode() >= toWriteVersion.versionCode()) {
+ // if the table version is greater than or equal to the write version,
then this is not an upgrade
+ LOG.warn("Table version {} is greater than or equal to write version {}.
No upgrade needed", fromTableVersion, toWriteVersion);
Review Comment:
As discussed, let's follow up to make sure the write client does not use a
write table version lower than the existing table version if auto upgrade is
disabled, to avoid corruption.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -85,19 +85,44 @@ public UpgradeDowngrade(
public static boolean needsUpgradeOrDowngrade(HoodieTableMetaClient
metaClient, HoodieWriteConfig config, HoodieTableVersion toWriteVersion) {
HoodieTableVersion fromTableVersion =
metaClient.getTableConfig().getTableVersion();
- return needsUpgrade(metaClient, config, toWriteVersion) ||
toWriteVersion.versionCode() < fromTableVersion.versionCode();
+ return needsUpgrade(metaClient, config, toWriteVersion) ||
needsDowngrade(fromTableVersion, toWriteVersion);
Review Comment:
@rahil-c let's track a follow-up JIRA to remove `needsUpgradeOrDowngrade`
and only call `needsUpgrade` or `needsDowngrade` from the caller to avoid
confusion, given that upgrade needs to check the auto-upgrade flag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
