[
https://issues.apache.org/jira/browse/IGNITE-25048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-25048:
-----------------------------------
Description:
The case is similar to https://issues.apache.org/jira/browse/IGNITE-25043, but
somewhat worse: we move configuration from one place to another. It's too hard
to introduce a flexible engine feature that would account for all possible
types of such changes, so additional coding will be required for every such an
occasion.
Anyway, please familiarize yourself with
https://issues.apache.org/jira/browse/IGNITE-25041, and its implementation if
it's available. This implementation will be similar.
h3. Implementation notes
I suggest supporting {{@Deprecated}} annotation for configuration properties.
We will be able to read deprecated values, but we won't show them to the user
anymore, and these values will be deleted from the configuration storage. The
difference between deprecated and deleted configuration is the following:
* We're still able to read the value of deprecated configuration in code
before it's lost forever.
This allows us to have a migration code for deprecated configurations, in which
we would copy old values to the new place. I propose the following algorithm:
We have two distinct cases where we could receive a value of deprecated
configuration, and handling them will be different in the implementation.
h4. 1. Configuration Storage
Here we should perform a transparent migration. When starting a node or
finishing join to the cluster, we should read "old" value. If it is not
default, we should run a migration routine, that would convert it to "new"
value, and set "default" to "old" value, thus making it effectively unused.
When working with these properties in the code, two options are possible:
* Configuration is local. We should only use "new" values, because
configuration has been migrated before we started our component.
* Configuration is global (cluster configuration). We should use "old" value
if it has non-default value, because we have a small window of time when
migration has not yet happened. If the value is default, we should use "new"
configuration. Same goes for configuration update listeners - we should have
them for both "old" and "new" configurations.
h4. 2. Dynamic configuration updates
Given that "old" values should no be present in configuration storages, we
can't propagate them via regular configuration lifecycle - they'll be lost in
such a case.
_Configuration must be migrated before it is written to the storage._
I suggest registering our migration routines in configuration changer instance
itself, so that every time user passes some configuration values to us, we:
* Parse it.
* Apply to current configuration tree.
* Also apply a migration routine to the same configuration tree ({*}new
step{*}).
* Send this update to configuration storage (proceed with regular flow).
I suppose we can identify modified sub-trees and this only run the routines
that match the specific subtrees, in order to not do these calculations
constantly.
I know it all sounds abstract. In speudo-code, I would like to see the
following:
{code:java}
MyConfigurationModule {
migrationRoutines() {
return Map.of(OldConfigurationSchema.class, superRoot -> {
var barValue = superRoot.foo().oldConfiguration().bar()
if (barValue != BAR_DEFAULT) {
// Not necessary I guess, it'll be deleted from the storage anyway.
// Let is stay here just to be an example.
superRoot.changeFoo().changeOldConfiguration().changeBar(BAR_DEFAULT)
superRoot.changeNewFoo().changeBar(barValue)
}
})
}
}{code}
The closure will be executed in a corresponding context automatically. I hope
that it clears my idea.
The migration code will always be written by a developer. If it needs to be
documented - we should document it.
h4. Deletion
Eventually, old configuration should be deleted from the code, with the
migration routine. This should only be done when we declare some versions as
incompatible, for example we could one day say {_}"you can't migrate directly
from 3.0 to 3.3 without an intermediate 3.2 upgrade"{_}, that would be fine I
think.
was:
The case is similar to https://issues.apache.org/jira/browse/IGNITE-25043, but
somewhat worse: we move configuration from one place to another. It's too hard
to introduce a flexible engine feature that would account for all possible
types of such changes, so additional coding will be required for every such an
occasion.
Anyway, please familiarize yourself with
https://issues.apache.org/jira/browse/IGNITE-25041, and its implementation if
it's available. This implementation will be similar.
h3. Implementation notes
I suggest supporting {{@Deprecated}} annotation for configuration properties.
We will be able to read deprecated values, but we won't show them to the user
anymore, and these values will be deleted from the configuration storage. The
difference between deprecated and deleted configuration is the following:
* We're still able to read the value of deprecated configuration in code
before it's lost forever.
This allows us to have a migration code for deprecated configurations, in which
we would copy old values to the new place. I propose the following algorithm:
We have two distinct cases where we could receive a value of deprecated
configuration, and handling them will be different in the implementation.
h4. 1. Configuration Storage
Here we should perform a transparent migration. When starting a node or
finishing join to the cluster, we should read "old" value. If it is not
default, we should run a migration routine, that would convert it to "new"
value, and set "default" to "old" value, thus making it effectively unused.
When working with these properties in the code, two options are possible:
* Configuration is local. We should only use "new" values, because
configuration has been migrated before we started our component.
* Configuration is global (cluster configuration). We should use "old" value
if it has non-default value, because we have a small window of time when
migration has not yet happened. If the value is default, we should use "new"
configuration. Same goes for configuration update listeners - we should have
them for both "old" and "new" configurations.
h4. 2. Dynamic configuration updates
Given that "old" values should no be present in configuration storages, we
can't propagate them via regular configuration lifecycle - they'll be lost in
such a case.
_Configuration must be migrated before it is written to the storage._
I suggest registering our migration routines in configuration changer instance
itself, so that every time user passes some configuration values to us, we:
* Parse it.
* Apply to current configuration tree.
* Also apply a migration routine to the same configuration tree ({*}new
step{*}).
* Send this update to configuration storage (proceed with regular flow).
I suppose we can identify modified sub-trees and this only run the routines
that match the specific subtrees, in order to not do these calculations
constantly.
I know it all sounds abstract. In speudo-code, I would like to see the
following:
{code:java}
MyConfigurationModule {
migrationRoutines() {
return Map.of(OldConfigurationSchema.class, superRoot -> {
var barValue = superRoot.foo().oldConfiguration().bar()
if (barValue != BAR_DEFAULT) {
superRoot.changeFoo().changeOldConfiguration().changeBar(BAR_DEFAULT)
superRoot.changeNewFoo().changeBar(barValue)
}
})
}
}{code}
The closure will be executed in a corresponding context automatically. I hope
that it clears my idea.
The migration code will always be written by a developer. If it needs to be
documented - we should document it.
h4. Deletion
Eventually, old configuration should be deleted from the code, with the
migration routine. This should only be done when we declare some versions as
incompatible, for example we could one day say {_}"you can't migrate directly
from 3.0 to 3.3 without an intermediate 3.2 upgrade"{_}, that would be fine I
think.
> Support configuration deprecation
> ---------------------------------
>
> Key: IGNITE-25048
> URL: https://issues.apache.org/jira/browse/IGNITE-25048
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
>
> The case is similar to https://issues.apache.org/jira/browse/IGNITE-25043,
> but somewhat worse: we move configuration from one place to another. It's too
> hard to introduce a flexible engine feature that would account for all
> possible types of such changes, so additional coding will be required for
> every such an occasion.
> Anyway, please familiarize yourself with
> https://issues.apache.org/jira/browse/IGNITE-25041, and its implementation if
> it's available. This implementation will be similar.
> h3. Implementation notes
> I suggest supporting {{@Deprecated}} annotation for configuration properties.
> We will be able to read deprecated values, but we won't show them to the user
> anymore, and these values will be deleted from the configuration storage. The
> difference between deprecated and deleted configuration is the following:
> * We're still able to read the value of deprecated configuration in code
> before it's lost forever.
> This allows us to have a migration code for deprecated configurations, in
> which we would copy old values to the new place. I propose the following
> algorithm:
> We have two distinct cases where we could receive a value of deprecated
> configuration, and handling them will be different in the implementation.
> h4. 1. Configuration Storage
> Here we should perform a transparent migration. When starting a node or
> finishing join to the cluster, we should read "old" value. If it is not
> default, we should run a migration routine, that would convert it to "new"
> value, and set "default" to "old" value, thus making it effectively unused.
> When working with these properties in the code, two options are possible:
> * Configuration is local. We should only use "new" values, because
> configuration has been migrated before we started our component.
> * Configuration is global (cluster configuration). We should use "old" value
> if it has non-default value, because we have a small window of time when
> migration has not yet happened. If the value is default, we should use "new"
> configuration. Same goes for configuration update listeners - we should have
> them for both "old" and "new" configurations.
> h4. 2. Dynamic configuration updates
> Given that "old" values should no be present in configuration storages, we
> can't propagate them via regular configuration lifecycle - they'll be lost in
> such a case.
> _Configuration must be migrated before it is written to the storage._
> I suggest registering our migration routines in configuration changer
> instance itself, so that every time user passes some configuration values to
> us, we:
> * Parse it.
> * Apply to current configuration tree.
> * Also apply a migration routine to the same configuration tree ({*}new
> step{*}).
> * Send this update to configuration storage (proceed with regular flow).
> I suppose we can identify modified sub-trees and this only run the routines
> that match the specific subtrees, in order to not do these calculations
> constantly.
> I know it all sounds abstract. In speudo-code, I would like to see the
> following:
>
> {code:java}
> MyConfigurationModule {
> migrationRoutines() {
> return Map.of(OldConfigurationSchema.class, superRoot -> {
> var barValue = superRoot.foo().oldConfiguration().bar()
> if (barValue != BAR_DEFAULT) {
> // Not necessary I guess, it'll be deleted from the storage anyway.
> // Let is stay here just to be an example.
> superRoot.changeFoo().changeOldConfiguration().changeBar(BAR_DEFAULT)
> superRoot.changeNewFoo().changeBar(barValue)
> }
> })
> }
> }{code}
> The closure will be executed in a corresponding context automatically. I hope
> that it clears my idea.
>
> The migration code will always be written by a developer. If it needs to be
> documented - we should document it.
> h4. Deletion
> Eventually, old configuration should be deleted from the code, with the
> migration routine. This should only be done when we declare some versions as
> incompatible, for example we could one day say {_}"you can't migrate directly
> from 3.0 to 3.3 without an intermediate 3.2 upgrade"{_}, that would be fine I
> think.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
