Mingliang Liu created FLINK-31984:
-------------------------------------
Summary: Savepoint on S3 should be relocatable if entropy
injection is not effective
Key: FLINK-31984
URL: https://issues.apache.org/jira/browse/FLINK-31984
Project: Flink
Issue Type: Improvement
Components: FileSystems, Runtime / Checkpointing
Affects Versions: 1.16.1
Reporter: Mingliang Liu
We have a limitation that if we create savepoints with an injected entropy,
they are not relocatable
(https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#triggering-savepoints).
FLINK-25952 improves the check by inspecting both the FileSystem extending
{{EntropyInjectingFileSystem}} and {{FlinkS3FileSystem#getEntropyInjectionKey}}
not returning null. We can improve this further by checking the checkpoint path
is indeed using the entropy injection key. Without that, the savepoint is not
relocatable even if the {{state.savepoints.dir}} does not contain the entropy.
In our setting, we enable entropy injection by setting {{s3.entropy.key}} to
{{\__ENTROPY_KEY\__}} and use the entropy key in the checkpoint path (for e.g.
{{s3://mybuket/checkpoints/__ENTROPY_KEY__/myapp}}). However, in the savepoint
path, we don't use the entropy key (for e.g. {{s3://mybuket/savepoints/myapp}})
because we want the savepoint to be relocatable. But the current logic still
generates non-relocatable savepoint path just because the entropy injection key
is non-null.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
