Re: Figuring out when a job has successfully restored state

2017-03-29 Thread Gyula Fóra
Thanks Till, This is exactly what I was looking for :) Gyula Till Rohrmann ezt írta (időpont: 2017. márc. 29., Sze, 10:23): > Hi Gyula, > > there exists a related issue [1]. Fixing this issue will move the state > restoration in the state DEPLOYING. This means that when you see a task > being i

Re: Figuring out when a job has successfully restored state

2017-03-29 Thread Till Rohrmann
Hi Gyula, there exists a related issue [1]. Fixing this issue will move the state restoration in the state DEPLOYING. This means that when you see a task being in state RUNNING, then it will have restored all of its eager state. [1] https://issues.apache.org/jira/browse/FLINK-4714 Cheers, Till

Re: Figuring out when a job has successfully restored state

2017-03-28 Thread Gyula Fóra
Hi, Another thought I had last night, maybe we could have another state for recovering jobs in the future. Deploying -> Recovering -> Running This recovering state might only be applicable for state backends that have to be restored before processing can start, lazy state backends (like external d

Figuring out when a job has successfully restored state

2017-03-27 Thread Gyula Fóra
Hi all, I am trying to figure out the best way to tell when a job has successfully restored all state and started process. My first idea was to check the rest api and the number of processed bytes for each parallel operator and if thats greater than 0, it started. Unfortunately this logic fails i