I just create a JIRA for this: https://issues.apache.org/jira/browse/FLINK-9633
On 06/21/2018 15:10，Chesnay Schepler<[email protected]> wrote： That's quite weird that it tries to us the local file-system. Maybe it derives the FS from state.backend.fs.checckpointdir, but uses it for savepoints.dir? What happens if you set state.backend.fs.checkpointdir also to HDFS? On 21.06.2018 08:07, Siew Wai Yow wrote: Thanks Chesnay, the application will take value from "state.savepoints.dir" as default if set target-directory to nul. But then it trying to create the directory in local machine, which caused the below error because it is a HDFS directory. The same URL works in previous Flink 1.3.2. Is something break in Flink 1.5.0? Or anything we need to extra configure? Thank you. Caused by: java.io.IOException: Mkdirs failed to create hdfs://192.168.56.150:8020/flink/savepoints/savepoint-855552-fb6eca4f3dbb at org.apache.flink.core.fs.local.LocalFileSystem.create(LocalFileSystem.java:271) at org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:121) at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:348) ... 25 more flink-conf.yaml: -------------------------------- state.backend: rocksdb state.backend.fs.checkpointdir: file:///tmp/rocksdb_simple_example state.backend.incremental: true state.backend.async: true state.checkpoints.num-retained: 5 state.savepoints.dir: hdfs://192.168.56.150:8020/flink/savepoints -Yow From: Chesnay Schepler <[email protected]> Sent: Wednesday, June 20, 2018 3:15 PM To:[email protected] Subject: Re: Questions regarding to Flink 1.5.0 REST API change I think you can set the target-directory to null. But I'm not sure why this particular request requires this, other request allow optional fields to simply be ommitted... On 20.06.2018 06:12, Siew Wai Yow wrote: Hi all, Seems pass in target-directory is a must now for checkpoints REST API, and the status will not response with save point directory anymore. I can pass in but the information is redundant with the same already defined in flink-config.yaml. May I know is there a way to retrieve the save point directory from flink-config.yaml in flink application? I am not able to get it from env.getConfig(). Thank you. From: Chesnay Schepler <[email protected]> Sent: Tuesday, June 19, 2018 11:55 PM To: [email protected] Subject: Re: Questions regarding to Flink 1.5.0 REST API change 1. PATCH to /jobs/:jobid, you can specify CANCEL/STOP with the "mode" query parameter 2. POST to /jobs/:jobid/savepoints, with a json payload. Returns a trigger id, used for 3). { "target-directory" : { "type" : "string" }, "cancel-job" : { "type" : "boolean" } } 3. GET to /jobs/:jobid/savepoints/:triggerid On 19.06.2018 17:40, Esteban Serrano wrote: For #1, you need to use a PATCH request to "/jobs/:jobid" On Tue, Jun 19, 2018 at 11:35 AM Siew Wai Yow <[email protected]> wrote: Hi, Regarding to Flink 1.5.0 REST API breaking change, The REST API to cancel a job was changed. The REST API to cancel a job with savepoint was changed. I have few dump questions, Any replacement for cancellation ONLY without save-point? Only found "/jobs/:jobid/savepoints". For "/jobs/:jobid/savepoints", how could I form the URL with cancellation and with directory afterward? http://192.168.56.151:8081/jobs/<jobid>/savepoints/cancel-job/true?? Any cancellation progress monitoring in 1.5 like previous version 1.3/1.4? previous version: /jobs/:jobid/cancel-with-savepoint/in-progress/:requestId Thank you. Regards, Yow
