Re: Trigerring Savepoint for the Flink Job

Sorry about the late reply. This reply is more specific to the Uber's AthenaX project . To trigger the savepoint we need to simply create an instance of YarnClusterClient. This class has implementation to trigger savepoint. To trigger the savepoint for any job

Re: Trigerring Savepoint for the Flink Job

Glad to know you found a solution. would you mind sharing you workaround in Flink 1.4.2. I am pretty sure other users would benefit from your effort too :-) Regarding the checkpoints / savepoints backend, we use HDFS as our stateful backend instead of RocksDB. We are working on putting that logic

Re: Trigerring Savepoint for the Flink Job

Just out of curiosity how do you save your checkpoint. Currently I'm using filesystem but I'm migrating it to RocksDB which allows for async operations to avoid latency at higher scale as we grow. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Trigerring Savepoint for the Flink Job

Hi Rongs. Thanks for your help. I was about to look into the CLI API but then I figured out how to trigger savepoint and restore a job's savepoint with the job running in detached mode. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Trigerring Savepoint for the Flink Job

Hi Anil, Glad to know that you upgrade the system to 1.4, from our experience there are quite a bit of changes requires to adapt to the new deployment model in 1.4 if I remember correctly. The Deployment model "run detach" in AthenaX does not support reattach back to the job, we use REST API to do

Re: Trigerring Savepoint for the Flink Job

Thanks for the reply Rong. We had updated Athenax to version 1.4. I had checked Flink 1.4, it's rest endpoint dose not support only creating Savepoint. It has cancel With Savepoint. I think creating Savepoint is supported in 1.5. Since we can't upgrade to 1.5 at the moment it would like to find a

Re: Trigerring Savepoint for the Flink Job

Hi Anil, I am actually one of the engineers maintaining Uber's AthenaX open-source platform. As for now, AthenaX is still running on Flink 1.3.2 so it might be weird to follow the 1.4.2 release document. (We are working on upgrading to latest 1.5 release) For your question regarding savepoints, A

Trigerring Savepoint for the Flink Job

I am using Flink 1.4.2. I have forker Uber's AthenaX project . The Flink jobs are deployed in Yarn cluster. I needed to save the Savepoint for all the jobs everyday. ClusterClient