RE: Basic questions about resuming stateful Flink jobs

2022-02-17 Thread Schwalbe Matthias
load a savepoint? Thanks, James. From: Piotr Nowojski mailto:pnowoj...@apache.org>> Sent: 16 February 2022 16:36 To: James Sandys-Lumsdaine mailto:jas...@hotmail.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Basic questions about resuming stateful F

Re: Basic questions about resuming stateful Flink jobs

2022-02-17 Thread Piotr Nowojski
lse can I create and >> recover from the save points? >> >> >> >> From what I’ve read there is state, checkpoints and save points – all of >> them hold state - and currently I can’t get any of these to restore when >> developing in an IDE and the program builds up

Re: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Cristian Constantinescu
ve points – all of > them hold state - and currently I can’t get any of these to restore when > developing in an IDE and the program builds up all state from scratch. So > what else do I need to do in my Java code to tell Flink to load a savepoint? > > > > Thanks, > > > >

RE: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Sandys-Lumsdaine, James
s up all state from scratch. So what else do I need to do in my Java code to tell Flink to load a savepoint? Thanks, James. From: Piotr Nowojski Sent: 16 February 2022 16:36 To: James Sandys-Lumsdaine Cc: user@flink.apache.org Subject: Re: Basic questions about resuming stateful Flink jobs CAU

Re: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Piotr Nowojski
Hi James, Sure! The basic idea of checkpoints is that they are fully owned by the running job and used for failure recovery. Thus by default if you stopped the job, checkpoints are being removed. If you want to stop a job and then later resume working from the same point that it has previously sto

Basic questions about resuming stateful Flink jobs

2022-02-16 Thread James Sandys-Lumsdaine
Hi all, I have a 1.14 Flink streaming workflow with many stateful functions that has a FsStateBackend and checkpointed enabled, although I haven't set a location for the checkpointed state. I've really struggled to understand how I can stop my Flink job and restart it and ensure it carries off