[
https://issues.apache.org/jira/browse/FLINK-40562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron He updated FLINK-40562:
-----------------------------
Description:
h2. This issue is being addressed together with FLINK-40559 in [Honor
--claimMode in application mode
apache/flink#29109|https://github.com/apache/flink/pull/29109].
h2. Problem
{{SavepointRestoreSettings.fromConfiguration()}} returns {{none()}} whenever
{{StateRecoveryOptions.SAVEPOINT_PATH}} is absent, even when
{{StateRecoveryOptions.RESTORE_MODE}} is explicitly configured.
The reconstructed settings therefore report the default {{NO_CLAIM}} instead of
an explicitly configured {{{}CLAIM{}}}. Serializing those settings into a fresh
{{Configuration}} also loses the explicit mode.
This is the configuration-reconstruction follow-up described in the "Related
but out of scope" section of FLINK-40559. It is separate from registering and
parsing the command-line options.
Session-mode submission is the motivating path, but the affected method is
shared code, not a session-only implementation. Other callers that reconstruct
settings from a configuration without a savepoint path can encounter the same
loss. This report does not establish the end-to-end impact for every deployment
mode.
h2. Minimal example
The behavior follows directly from the current implementation and is explicitly
asserted by the existing {{{}testFromConfigurationWithNoPath{}}}. Source
inspected on {{master}} at
[3593f080f09fb9f974dca62279a98cc16505acb9|https://github.com/apache/flink/commit/3593f080f09fb9f974dca62279a98cc16505acb9].
This report does not claim an end-to-end cluster reproduction.
{noformat}
Configuration input = new Configuration();
input.set(StateRecoveryOptions.RESTORE_MODE, RecoveryClaimMode.CLAIM);
// No StateRecoveryOptions.SAVEPOINT_PATH.
SavepointRestoreSettings settings =
SavepointRestoreSettings.fromConfiguration(input);
settings.restoreSavepoint(); // false
settings.getRecoveryClaimMode(); // NO_CLAIM, although input contains CLAIM
Configuration output = new Configuration();
SavepointRestoreSettings.toConfiguration(settings, output);
output.containsKey(StateRecoveryOptions.RESTORE_MODE.key()); // false
{noformat}
The input configuration is not modified. The loss occurs in the reconstructed
settings and in a subsequent serialization to a fresh configuration.
h2. Submission-path relevance
After the parser changes proposed in
[apache/flink#29109|https://github.com/apache/flink/pull/29109] for FLINK-40559:
# {{ProgramOptions}} can hold a claim mode without a savepoint path.
# {{ProgramOptions.applyToConfiguration()}} serializes that explicit mode.
# {{ExecutionConfigAccessor.getSavepointRestoreSettings()}} calls
{{SavepointRestoreSettings.fromConfiguration()}} and loses the mode because the
path is absent.
# {{PipelineExecutorUtils}} uses the reconstructed settings when preparing the
{{{}StreamGraph{}}}.
An illustrative CLI input for this path is:
{noformat}
bin/flink run --claimMode CLAIM job.jar
{noformat}
On master before FLINK-40559, the parser can already drop the pathless option;
that is a separate, earlier loss. The direct configuration example above
isolates this issue without depending on the parser fix.
h2. Proposed behavior and scope
Preserve an explicitly configured recovery claim mode when reconstructing
settings without a savepoint path, while keeping {{restoreSavepoint()}} false
and {{getRestorePath()}} null. The pathless factory proposed in FLINK-40559
could be reused once available.
Preserve the explicit/unset distinction introduced by FLINK-39673:
* Explicit {{CLAIM}} and explicit {{NO_CLAIM}} must both survive
reconstruction and serialization into a fresh configuration.
* An absent claim mode must not become an explicitly written default.
* An absent {{SAVEPOINT_IGNORE_UNCLAIMED_STATE}} must not become an explicitly
written false value.
* Existing with-savepoint behavior must remain unchanged.
The existing {{testFromConfigurationWithNoPath}} deliberately expects
{{none()}} despite a configured {{{}CLAIM{}}}. This proposal changes that
contract, not just an untested branch. Maintainer agreement is needed on
supporting pathless claim mode in general configuration consumers and session
submission. Retaining the value alone does not establish that every execution
path uses it during HA recovery.
The intended scope is configuration reconstruction and propagation, not
changing checkpoint ownership rules or making a new job restore state without a
restore source.
h2. Suggested coverage
* Runtime configuration round trips without a path, covering explicit
{{{}CLAIM{}}}, explicit {{{}NO_CLAIM{}}}, and absent mode; inspect key presence
as well as effective values.
* Preserve the existing explicit/unset semantics for the
allow-non-restored-state option.
* Client coverage through {{ProgramOptions}} and
{{{}ExecutionConfigAccessor{}}}, rather than only invoking the CLI parser
directly, after FLINK-40559.
* Retain coverage for settings with a savepoint path.
h2. Related issues and duplicate search
Searched Apache Flink Jira across all statuses on 2026-09-05 using claimMode,
restoreMode, "claim mode", "restore mode", SavepointRestoreSettings,
ExecutionConfigAccessor, the claim-mode configuration key, and
pathless/fromConfiguration terms. No separate exact duplicate was found; this
is not a guarantee that no differently worded report exists.
* FLINK-40559 (Open): parent context; explicitly excludes this reconstruction
issue.
* FLINK-26316 (Open): dynamic savepoint configuration overwritten by
{{none()}} during {{{}ProgramOptions.applyToConfiguration(){}}}, rather than a
configured mode discarded during reconstruction.
* FLINK-34015 (Open): dynamic properties overwritten by CLI defaults; its
discussion describes a restore path supplied via dynamic configuration.
* FLINK-39673 (Closed, Fixed): prevents unspecified defaults from overriding
user configuration during serialization. Its explicit/unset semantics must be
preserved.
* FLINK-28651 (Closed, Fixed): REST {{JarRunHandler}} choosing a default
instead of the configured restore mode, a different consumer and failure point.
h2. Source references
* [SavepointRestoreSettings.fromConfiguration() and
toConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettings.java]
* [Existing
testFromConfigurationWithNoPath|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/test/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettingsTest.java]
*
[ProgramOptions.applyToConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptions.java#L179-L189]
*
[ExecutionConfigAccessor.getSavepointRestoreSettings()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ExecutionConfigAccessor.java#L93-L95]
* [PipelineExecutorUtils propagation to
StreamGraph|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/deployment/executors/PipelineExecutorUtils.java#L107-L110]
was:
h2. This issue is being addressed together with FLINK-40559 in
[[FLINK-40559][clients] Honor --claimMode in application mode
apache/flink#29109|https://github.com/apache/flink/pull/29109].
h2. Problem
{{SavepointRestoreSettings.fromConfiguration()}} returns {{none()}} whenever
{{StateRecoveryOptions.SAVEPOINT_PATH}} is absent, even when
{{StateRecoveryOptions.RESTORE_MODE}} is explicitly configured.
The reconstructed settings therefore report the default {{NO_CLAIM}} instead of
an explicitly configured {{{}CLAIM{}}}. Serializing those settings into a fresh
{{Configuration}} also loses the explicit mode.
This is the configuration-reconstruction follow-up described in the "Related
but out of scope" section of FLINK-40559. It is separate from registering and
parsing the command-line options.
Session-mode submission is the motivating path, but the affected method is
shared code, not a session-only implementation. Other callers that reconstruct
settings from a configuration without a savepoint path can encounter the same
loss. This report does not establish the end-to-end impact for every deployment
mode.
h2. Minimal example
The behavior follows directly from the current implementation and is explicitly
asserted by the existing {{{}testFromConfigurationWithNoPath{}}}. Source
inspected on {{master}} at
[3593f080f09fb9f974dca62279a98cc16505acb9|https://github.com/apache/flink/commit/3593f080f09fb9f974dca62279a98cc16505acb9].
This report does not claim an end-to-end cluster reproduction.
{noformat}
Configuration input = new Configuration();
input.set(StateRecoveryOptions.RESTORE_MODE, RecoveryClaimMode.CLAIM);
// No StateRecoveryOptions.SAVEPOINT_PATH.
SavepointRestoreSettings settings =
SavepointRestoreSettings.fromConfiguration(input);
settings.restoreSavepoint(); // false
settings.getRecoveryClaimMode(); // NO_CLAIM, although input contains CLAIM
Configuration output = new Configuration();
SavepointRestoreSettings.toConfiguration(settings, output);
output.containsKey(StateRecoveryOptions.RESTORE_MODE.key()); // false
{noformat}
The input configuration is not modified. The loss occurs in the reconstructed
settings and in a subsequent serialization to a fresh configuration.
h2. Submission-path relevance
After the parser changes proposed in
[apache/flink#29109|https://github.com/apache/flink/pull/29109] for FLINK-40559:
# {{ProgramOptions}} can hold a claim mode without a savepoint path.
# {{ProgramOptions.applyToConfiguration()}} serializes that explicit mode.
# {{ExecutionConfigAccessor.getSavepointRestoreSettings()}} calls
{{SavepointRestoreSettings.fromConfiguration()}} and loses the mode because the
path is absent.
# {{PipelineExecutorUtils}} uses the reconstructed settings when preparing the
{{{}StreamGraph{}}}.
An illustrative CLI input for this path is:
{noformat}
bin/flink run --claimMode CLAIM job.jar
{noformat}
On master before FLINK-40559, the parser can already drop the pathless option;
that is a separate, earlier loss. The direct configuration example above
isolates this issue without depending on the parser fix.
h2. Proposed behavior and scope
Preserve an explicitly configured recovery claim mode when reconstructing
settings without a savepoint path, while keeping {{restoreSavepoint()}} false
and {{getRestorePath()}} null. The pathless factory proposed in FLINK-40559
could be reused once available.
Preserve the explicit/unset distinction introduced by FLINK-39673:
* Explicit {{CLAIM}} and explicit {{NO_CLAIM}} must both survive
reconstruction and serialization into a fresh configuration.
* An absent claim mode must not become an explicitly written default.
* An absent {{SAVEPOINT_IGNORE_UNCLAIMED_STATE}} must not become an explicitly
written false value.
* Existing with-savepoint behavior must remain unchanged.
The existing {{testFromConfigurationWithNoPath}} deliberately expects
{{none()}} despite a configured {{{}CLAIM{}}}. This proposal changes that
contract, not just an untested branch. Maintainer agreement is needed on
supporting pathless claim mode in general configuration consumers and session
submission. Retaining the value alone does not establish that every execution
path uses it during HA recovery.
The intended scope is configuration reconstruction and propagation, not
changing checkpoint ownership rules or making a new job restore state without a
restore source.
h2. Suggested coverage
* Runtime configuration round trips without a path, covering explicit
{{{}CLAIM{}}}, explicit {{{}NO_CLAIM{}}}, and absent mode; inspect key presence
as well as effective values.
* Preserve the existing explicit/unset semantics for the
allow-non-restored-state option.
* Client coverage through {{ProgramOptions}} and
{{{}ExecutionConfigAccessor{}}}, rather than only invoking the CLI parser
directly, after FLINK-40559.
* Retain coverage for settings with a savepoint path.
h2. Related issues and duplicate search
Searched Apache Flink Jira across all statuses on 2026-09-05 using claimMode,
restoreMode, "claim mode", "restore mode", SavepointRestoreSettings,
ExecutionConfigAccessor, the claim-mode configuration key, and
pathless/fromConfiguration terms. No separate exact duplicate was found; this
is not a guarantee that no differently worded report exists.
* FLINK-40559 (Open): parent context; explicitly excludes this reconstruction
issue.
* FLINK-26316 (Open): dynamic savepoint configuration overwritten by
{{none()}} during {{{}ProgramOptions.applyToConfiguration(){}}}, rather than a
configured mode discarded during reconstruction.
* FLINK-34015 (Open): dynamic properties overwritten by CLI defaults; its
discussion describes a restore path supplied via dynamic configuration.
* FLINK-39673 (Closed, Fixed): prevents unspecified defaults from overriding
user configuration during serialization. Its explicit/unset semantics must be
preserved.
* FLINK-28651 (Closed, Fixed): REST {{JarRunHandler}} choosing a default
instead of the configured restore mode, a different consumer and failure point.
h2. Source references
* [SavepointRestoreSettings.fromConfiguration() and
toConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettings.java]
* [Existing
testFromConfigurationWithNoPath|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/test/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettingsTest.java]
*
[ProgramOptions.applyToConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptions.java#L179-L189]
*
[ExecutionConfigAccessor.getSavepointRestoreSettings()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ExecutionConfigAccessor.java#L93-L95]
* [PipelineExecutorUtils propagation to
StreamGraph|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/deployment/executors/PipelineExecutorUtils.java#L107-L110]
> Preserve pathless claim mode across configuration round trips
> -------------------------------------------------------------
>
> Key: FLINK-40562
> URL: https://issues.apache.org/jira/browse/FLINK-40562
> Project: Flink
> Issue Type: Improvement
> Components: Client / Job Submission
> Reporter: Aaron He
> Priority: Minor
>
> h2. This issue is being addressed together with FLINK-40559 in [Honor
> --claimMode in application mode
> apache/flink#29109|https://github.com/apache/flink/pull/29109].
> h2. Problem
> {{SavepointRestoreSettings.fromConfiguration()}} returns {{none()}} whenever
> {{StateRecoveryOptions.SAVEPOINT_PATH}} is absent, even when
> {{StateRecoveryOptions.RESTORE_MODE}} is explicitly configured.
> The reconstructed settings therefore report the default {{NO_CLAIM}} instead
> of an explicitly configured {{{}CLAIM{}}}. Serializing those settings into a
> fresh {{Configuration}} also loses the explicit mode.
> This is the configuration-reconstruction follow-up described in the "Related
> but out of scope" section of FLINK-40559. It is separate from registering and
> parsing the command-line options.
> Session-mode submission is the motivating path, but the affected method is
> shared code, not a session-only implementation. Other callers that
> reconstruct settings from a configuration without a savepoint path can
> encounter the same loss. This report does not establish the end-to-end impact
> for every deployment mode.
> h2. Minimal example
> The behavior follows directly from the current implementation and is
> explicitly asserted by the existing {{{}testFromConfigurationWithNoPath{}}}.
> Source inspected on {{master}} at
> [3593f080f09fb9f974dca62279a98cc16505acb9|https://github.com/apache/flink/commit/3593f080f09fb9f974dca62279a98cc16505acb9].
> This report does not claim an end-to-end cluster reproduction.
> {noformat}
> Configuration input = new Configuration();
> input.set(StateRecoveryOptions.RESTORE_MODE, RecoveryClaimMode.CLAIM);
> // No StateRecoveryOptions.SAVEPOINT_PATH.
> SavepointRestoreSettings settings =
> SavepointRestoreSettings.fromConfiguration(input);
> settings.restoreSavepoint(); // false
> settings.getRecoveryClaimMode(); // NO_CLAIM, although input contains CLAIM
> Configuration output = new Configuration();
> SavepointRestoreSettings.toConfiguration(settings, output);
> output.containsKey(StateRecoveryOptions.RESTORE_MODE.key()); // false
> {noformat}
> The input configuration is not modified. The loss occurs in the reconstructed
> settings and in a subsequent serialization to a fresh configuration.
> h2. Submission-path relevance
> After the parser changes proposed in
> [apache/flink#29109|https://github.com/apache/flink/pull/29109] for
> FLINK-40559:
> # {{ProgramOptions}} can hold a claim mode without a savepoint path.
> # {{ProgramOptions.applyToConfiguration()}} serializes that explicit mode.
> # {{ExecutionConfigAccessor.getSavepointRestoreSettings()}} calls
> {{SavepointRestoreSettings.fromConfiguration()}} and loses the mode because
> the path is absent.
> # {{PipelineExecutorUtils}} uses the reconstructed settings when preparing
> the {{{}StreamGraph{}}}.
> An illustrative CLI input for this path is:
> {noformat}
> bin/flink run --claimMode CLAIM job.jar
> {noformat}
> On master before FLINK-40559, the parser can already drop the pathless
> option; that is a separate, earlier loss. The direct configuration example
> above isolates this issue without depending on the parser fix.
> h2. Proposed behavior and scope
> Preserve an explicitly configured recovery claim mode when reconstructing
> settings without a savepoint path, while keeping {{restoreSavepoint()}} false
> and {{getRestorePath()}} null. The pathless factory proposed in FLINK-40559
> could be reused once available.
> Preserve the explicit/unset distinction introduced by FLINK-39673:
> * Explicit {{CLAIM}} and explicit {{NO_CLAIM}} must both survive
> reconstruction and serialization into a fresh configuration.
> * An absent claim mode must not become an explicitly written default.
> * An absent {{SAVEPOINT_IGNORE_UNCLAIMED_STATE}} must not become an
> explicitly written false value.
> * Existing with-savepoint behavior must remain unchanged.
> The existing {{testFromConfigurationWithNoPath}} deliberately expects
> {{none()}} despite a configured {{{}CLAIM{}}}. This proposal changes that
> contract, not just an untested branch. Maintainer agreement is needed on
> supporting pathless claim mode in general configuration consumers and session
> submission. Retaining the value alone does not establish that every execution
> path uses it during HA recovery.
> The intended scope is configuration reconstruction and propagation, not
> changing checkpoint ownership rules or making a new job restore state without
> a restore source.
> h2. Suggested coverage
> * Runtime configuration round trips without a path, covering explicit
> {{{}CLAIM{}}}, explicit {{{}NO_CLAIM{}}}, and absent mode; inspect key
> presence as well as effective values.
> * Preserve the existing explicit/unset semantics for the
> allow-non-restored-state option.
> * Client coverage through {{ProgramOptions}} and
> {{{}ExecutionConfigAccessor{}}}, rather than only invoking the CLI parser
> directly, after FLINK-40559.
> * Retain coverage for settings with a savepoint path.
> h2. Related issues and duplicate search
> Searched Apache Flink Jira across all statuses on 2026-09-05 using claimMode,
> restoreMode, "claim mode", "restore mode", SavepointRestoreSettings,
> ExecutionConfigAccessor, the claim-mode configuration key, and
> pathless/fromConfiguration terms. No separate exact duplicate was found; this
> is not a guarantee that no differently worded report exists.
> * FLINK-40559 (Open): parent context; explicitly excludes this
> reconstruction issue.
> * FLINK-26316 (Open): dynamic savepoint configuration overwritten by
> {{none()}} during {{{}ProgramOptions.applyToConfiguration(){}}}, rather than
> a configured mode discarded during reconstruction.
> * FLINK-34015 (Open): dynamic properties overwritten by CLI defaults; its
> discussion describes a restore path supplied via dynamic configuration.
> * FLINK-39673 (Closed, Fixed): prevents unspecified defaults from overriding
> user configuration during serialization. Its explicit/unset semantics must be
> preserved.
> * FLINK-28651 (Closed, Fixed): REST {{JarRunHandler}} choosing a default
> instead of the configured restore mode, a different consumer and failure
> point.
> h2. Source references
> * [SavepointRestoreSettings.fromConfiguration() and
> toConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettings.java]
> * [Existing
> testFromConfigurationWithNoPath|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-runtime/src/test/java/org/apache/flink/runtime/jobgraph/SavepointRestoreSettingsTest.java]
> *
> [ProgramOptions.applyToConfiguration()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptions.java#L179-L189]
> *
> [ExecutionConfigAccessor.getSavepointRestoreSettings()|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/cli/ExecutionConfigAccessor.java#L93-L95]
> * [PipelineExecutorUtils propagation to
> StreamGraph|https://github.com/apache/flink/blob/3593f080f09fb9f974dca62279a98cc16505acb9/flink-clients/src/main/java/org/apache/flink/client/deployment/executors/PipelineExecutorUtils.java#L107-L110]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
