[
https://issues.apache.org/jira/browse/FLINK-31593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708612#comment-17708612
]
Roman Khachatryan commented on FLINK-31593:
-------------------------------------------
[~mapohl] I looked into StatefulJobWBroadcastStateMigrationITCase failiure
which fails CI for your PRs.
It failed because TM was unable to find the referenced sst file:
{code:java}
java.io.FileNotFoundException:
/tmp/junit6895540657846214402/checkpoints_e6e27005-a3fb-4c23-81e8-f6a2e827079b/4b3009354700b5a3a966f015556c8fb2/taskowned/1d9945c1-44b9-4d7c-9b20-d5ff90e68d7b
(No such file or directory) {code}
The file is indeed referenced in _metadata and is missing in resources
(stateful-scala-with-broadcast-udf-migration-itcase-flink1.17-rocksdb-checkpoint/_metadata).
It was likely generated during changelog materialization (becuase it uses
taskowned/ dir). Materialization is not deterministic, which might explain why
it didn't show up earlier.
It should be
[possible|https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/ops/state/state_backends/#upgrading-existing-jobs]
to restore from such a checkpoint (given all the files are present) even with
changelog disabled.
Alternatively, we can just generate the checkpoints with changelog (or
materialization) disabled.
Am I missing something?
(if the above sounds reasonable, [~mapohl] please let me know if I can help in
checkpoints generation)
> Update reference data for Migration Tests
> -----------------------------------------
>
> Key: FLINK-31593
> URL: https://issues.apache.org/jira/browse/FLINK-31593
> Project: Flink
> Issue Type: Sub-task
> Reporter: Matthias Pohl
> Assignee: Matthias Pohl
> Priority: Major
> Labels: pull-request-available
> Attachments:
> FLINK-31593.StatefulJobSavepointMigrationITCase.create_snapshot.log,
> FLINK-31593.StatefulJobSavepointMigrationITCase.verify_snapshot.log
>
>
> # Update {{CURRENT_VERSION in TypeSerializerUpgradeTestBase}} with the new
> version. This will likely fail some tests because snapshots are missing for
> that version. Generate them, for example in
> {{TypeSerializerUpgradeTestBase.}}
> # (major/minor only) Update migration tests in master to cover migration
> from new version: (search for usages of FlinkV{{{}ersion{}}})
> ** AbstractOperatorRestoreTestBase
> ** CEPMigrationTest
> ** BucketingSinkMigrationTest
> ** FlinkKafkaConsumerBaseMigrationTest
> ** ContinuousFileProcessingMigrationTest
> ** WindowOperatorMigrationTest
> ** StatefulJobSavepointMigrationITCase
> ** StatefulJobWBroadcastStateMigrationITCase
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
