[
https://issues.apache.org/jira/browse/HUDI-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-9702:
--------------------------------------
Description:
[https://github.com/apache/hudi/pull/13519#discussion_r2263549244]
During upgrade/downgrade, there are two sets of configurations: toUpdate and
toRemove.
There could be some corners that a property is update/added, and then removed
later, but may be added again. If we don't anything, we could see that the
configuration will exist in both toUpdate and toRemove config sets.
Therefore, we need to make this logic straight.
Ethan's comment:
{code:java}
If the table is upgraded from 6 to 9, when executing the eight-to-nine upgrade
handler, based on the logic the table config fetched here is still from
original version 6, because the table config is only rewritten after all
upgrade is called. Does the logic here assume that some table configs are
already updated to table version 8? I think this assumption make senses for
maintaining the upgrade logic, but it requires changing the logic to follow
that, i.e., passing in the updated and deleted configs from 6->8 upgrades.
{code}
was:
https://github.com/apache/hudi/pull/13519#discussion_r2263549244
During upgrade/downgrade, there are two sets of configurations: toUpdate and
toRemove.
There could be some corners that a property is update/added, and then removed
later, but may be added again. If we don't anything, we could see that the
configuration will exist in both toUpdate and toRemove config sets.
Therefore, we need to make this logic straight.
> Unify configurations for update and delete during upgrade/downgrade
> -------------------------------------------------------------------
>
> Key: HUDI-9702
> URL: https://issues.apache.org/jira/browse/HUDI-9702
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Lin Liu
> Assignee: Lin Liu
> Priority: Critical
> Fix For: 1.1.0
>
>
> [https://github.com/apache/hudi/pull/13519#discussion_r2263549244]
> During upgrade/downgrade, there are two sets of configurations: toUpdate and
> toRemove.
> There could be some corners that a property is update/added, and then removed
> later, but may be added again. If we don't anything, we could see that the
> configuration will exist in both toUpdate and toRemove config sets.
> Therefore, we need to make this logic straight.
> Ethan's comment:
> {code:java}
> If the table is upgraded from 6 to 9, when executing the eight-to-nine
> upgrade handler, based on the logic the table config fetched here is still
> from original version 6, because the table config is only rewritten after all
> upgrade is called. Does the logic here assume that some table configs are
> already updated to table version 8? I think this assumption make senses for
> maintaining the upgrade logic, but it requires changing the logic to follow
> that, i.e., passing in the updated and deleted configs from 6->8 upgrades.
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
