Hi Aviad,
I had a similar situation and my solution was to use the flink
monitoring rest api (/jobs/{jobid}/checkpoints) to get the mapping
between job and checkpoint file.
Wrap this in a script and run periodically( in my case, it was 30 sec).
You can also configure each job with an externalized checkpoint
directory. Refer
https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/checkpoints.html#directory-structure
Thanks,
Jins
On 11/15/2017 06:34 AM, Aviad Rotem wrote:
Hi,
I have several jobs which configured for external check-pointing
(enableExternalizedCheckpoints)
how can I correlate between checkpoint and jobs.
for example, if i want to write script which monitor if the job is up
or not and if the job is down it will resume the job from the
externalized checkpoint.
how could i know which checkpoint belong to the specific job?
can I configure each job to write the external check-pointing to a
different position?
my configuration is:
*state.backend*: rocksdb
*state.backend.fs.checkpointdir*: s3a://flink-bucket/backend/checkpoints
*state.checkpoints.dir*: s3a://flink-bucket/checkpoints
and in the code I set:
enableCheckpointing
enableExternalizedCheckpoints
