Thanks Chesnay, the application will take value from "state.savepoints.dir" as
default if set target-directory to nul. But then it trying to create the
directory in local machine, which caused the below error because it is a HDFS
directory. The same URL works in previous Flink 1.3.2. Is something break in
Flink 1.5.0? Or anything we need to extra configure? Thank you.
Caused by: java.io.IOException: Mkdirs failed to create
hdfs://192.168.56.150:8020/flink/savepoints/savepoint-855552-fb6eca4f3dbb
at
org.apache.flink.core.fs.local.LocalFileSystem.create(LocalFileSystem.java:271)
at
org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:121)
at
org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:348)
... 25 more
flink-conf.yaml:
--------------------------------
state.backend: rocksdb
state.backend.fs.checkpointdir: file:///tmp/rocksdb_simple_example
state.backend.incremental: true
state.backend.async: true
state.checkpoints.num-retained: 5
state.savepoints.dir: hdfs://192.168.56.150:8020/flink/savepoints
-Yow
________________________________
From: Chesnay Schepler <[email protected]>
Sent: Wednesday, June 20, 2018 3:15 PM
To: [email protected]
Subject: Re: Questions regarding to Flink 1.5.0 REST API change
I think you can set the target-directory to null. But I'm not sure why this
particular request requires this, other request allow optional fields to simply
be ommitted...
On 20.06.2018 06:12, Siew Wai Yow wrote:
Hi all,
Seems pass in target-directory is a must now for checkpoints REST API, and the
status will not response with save point directory anymore. I can pass in but
the information is redundant with the same already defined in
flink-config.yaml. May I know is there a way to retrieve the save point
directory from flink-config.yaml in flink application? I am not able to get it
from env.getConfig(). Thank you.
________________________________
From: Chesnay Schepler <[email protected]><mailto:[email protected]>
Sent: Tuesday, June 19, 2018 11:55 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: Questions regarding to Flink 1.5.0 REST API change
1. PATCH to /jobs/:jobid, you can specify CANCEL/STOP with the "mode" query
parameter
2. POST to /jobs/:jobid/savepoints, with a json payload. Returns a trigger id,
used for 3).
{
"target-directory" : {
"type" : "string"
},
"cancel-job" : {
"type" : "boolean"
}
}
3. GET to /jobs/:jobid/savepoints/:triggerid
On 19.06.2018 17:40, Esteban Serrano wrote:
For #1, you need to use a PATCH request to "/jobs/:jobid"
On Tue, Jun 19, 2018 at 11:35 AM Siew Wai Yow
<[email protected]<mailto:[email protected]>> wrote:
Hi,
Regarding to Flink 1.5.0 REST API breaking change,
* The REST API to cancel a job was changed.
* The REST API to cancel a job with savepoint was changed.
I have few dump questions,
1. Any replacement for cancellation ONLY without save-point? Only found
"/jobs/:jobid/savepoints".
2. For "/jobs/:jobid/savepoints", how could I form the URL with cancellation
and with directory afterward?
* http://192.168.56.151:8081/jobs/<jobid>/savepoints/cancel-job/true??
3. Any cancellation progress monitoring in 1.5 like previous version 1.3/1.4?
* previous version:
/jobs/:jobid/cancel-with-savepoint/in-progress/:requestId
Thank you.
Regards,
Yow
