Re: Redeployements and state

2016-02-05 Thread Stephan Ewen
@Don In Flink, the client needs not keep running, it can detach itself once the job is submitted, or stay connected and receive status messages. Since the client is not necessary for the program execution, I think you don't need "supervise" or anything like that... On Mon, Feb 1, 2016 at 7:23 PM,

Re: Redeployements and state

2016-02-01 Thread Ufuk Celebi
> On 01 Feb 2016, at 17:14, Don Frascuchon wrote: > > Hi, > > In reference with this topic, there is any feature for automatically restart > job after a task exception? Like --supervise command line option in apache > spark If you are referring to job manager/task manager instances: No. Cur

Re: Redeployements and state

2016-02-01 Thread Don Frascuchon
Hi, In reference with this topic, there is any feature for automatically restart job after a task exception? Like --supervise command line option in apache spark Thanks in advance! El mar., 26 ene. 2016 a las 11:07, Ufuk Celebi () escribió: > Hey Niels! > > Stephan gave a very good summary of

Re: Redeployements and state

2016-01-26 Thread Ufuk Celebi
Hey Niels! Stephan gave a very good summary of the current state of things. What do you think of the outlined stop with savepoint method? Regarding the broken links: I’ve fixed various broken links in the master docs yesterday. If you encounter something again, feel free to post it to the ML or

Re: Redeployements and state

2016-01-25 Thread Stephan Ewen
Hi Niels! There is a slight mismatch between your thoughts and the current design, but not much. What you describe (at the start of the job, the latest checkpoint is automatically loaded) is basically what the high-availability setup does if the master dies. The new master loads all jobs and cont

Re: Redeployements and state

2016-01-22 Thread Niels Basjes
Hi, @Max: Thanks for the new URL. I noticed that a lot (in fact almost all) of links in the new manuals lead to 404 errors. Maybe you should run an automated test to find them all. I did a bit of reading about the savepoints and that in fact they are written as "Allow to trigger checkpoints manua

Re: Redeployements and state

2016-01-18 Thread Maximilian Michels
The documentation layout changed in the master. Then new URL: https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/savepoints.html On Thu, Jan 14, 2016 at 2:21 PM, Niels Basjes wrote: > Yes, that is exactly the type of solution I was looking for. > > I'll dive into this. > Thanks

Re: Redeployements and state

2016-01-14 Thread Niels Basjes
Yes, that is exactly the type of solution I was looking for. I'll dive into this. Thanks guys! Niels On Thu, Jan 14, 2016 at 11:55 AM, Ufuk Celebi wrote: > Hey Niels, > > as Gabor wrote, this feature has been merged to the master branch recently. > > The docs are online here: > https://ci.apac

Re: Redeployements and state

2016-01-14 Thread Ufuk Celebi
Hey Niels, as Gabor wrote, this feature has been merged to the master branch recently. The docs are online here: https://ci.apache.org/projects/flink/flink-docs-master/apis/savepoints.html Feel free to report back your experience with it if you give it a try. – Ufuk > On 14 Jan 2016, at 11:09

Re: Redeployements and state

2016-01-14 Thread Gábor Gévay
Hello, You are probably looking for this feature: https://issues.apache.org/jira/browse/FLINK-2976 Best, Gábor 2016-01-14 11:05 GMT+01:00 Niels Basjes : > Hi, > > I'm working on a streaming application using Flink. > Several steps in the processing are state-full (I use custom Windows and > s

Redeployements and state

2016-01-14 Thread Niels Basjes
Hi, I'm working on a streaming application using Flink. Several steps in the processing are state-full (I use custom Windows and state-full operators ). Now if during a normal run an worker fails the checkpointing system will be used to recover. But what if the entire application is stopped (del