Re: Fast restart of a job with a large state

2019-04-24 Thread Sergey Zhemzhitsky
g > around in the community but nothing concrete yet. > > Cheers, > Till > > On Tue, Apr 23, 2019 at 6:58 PM Sergey Zhemzhitsky > wrote: > >> Hi Stefan, Paul, >> >> Thanks for the tips! Currently I have not tried neither rescaling from >> checkpoints n

Re: Fast restart of a job with a large state

2019-04-23 Thread Sergey Zhemzhitsky
nk-docs-stable/ops/state/large_state_tuning.html#task-local-recovery > > Best, > Paul Lam > > 在 2019年4月18日,18:04,Paul Lam 写道: > > Hi, > > Have you tried task local recovery [1]? > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoi

Fast restart of a job with a large state

2019-04-17 Thread Sergey Zhemzhitsky
Hi Flinkers, Operating different flink jobs I've discovered that job restarts with a pretty large state (in my case this is up to 100GB+) take quite a lot of time. For example, to restart a job (e.g. to update it) the savepoint is created, and in case of savepoints all the state seems to be pushed

Mapping of CLI arguments to flink-conf.yaml

2019-03-09 Thread Sergey Zhemzhitsky
Hi there I mostly use flink for stream-processing jobs and deploy them as independent ones [1]. So I'd like every job to have its own configuration and don't like to remember the values of all the command line arguments used to start the job every time. In spark for most of the CLI arguments ther