Re: Missing checkpoint when restarting failed job

2017-11-28 Thread Gerard Garcia
I've been monitoring the task and checkpoint 1 never gets deleted. Right now we have: chk-1 chk-1222 chk-326 chk-329 chk-357 chk-358 chk-8945 chk-8999 chk-9525 chk-9788 chk-9789 chk-9790 chk-9791 I made the task fail and it recovered without problems so for now I would say that the pro

Re: Missing checkpoint when restarting failed job

2017-11-21 Thread Stefan Richter
Ok, thanks for trying to reproduce this. If possible, could you also activate trace-level logging for class org.apache.flink.runtime.state.SharedStateRegistry? In case the problem occurs, this would greatly help to understand what was going on. > Am 21.11.2017 um 15:16 schrieb gerardg : > >> w

Re: Missing checkpoint when restarting failed job

2017-11-21 Thread gerardg
> where exactly did you read many times that incremental checkpoints cannot reference files from previous > checkpoints, because we would have to correct that information. In fact, > this is how incremental checkpoints work. My fault, I read it in some other posts in the mailing list but now tha

Re: Missing checkpoint when restarting failed job

2017-11-21 Thread Stefan Richter
Hi, where exactly did you read many times that incremental checkpoints cannot reference files from previous checkpoints, because we would have to correct that information. In fact, this is how incremental checkpoints work. Now for this case, I would consider it extremely unlikely that a checkpo