Re: Adding a new operator

2015-04-26 Thread Markus Holzemer
Hey Andrea, perhaps you are looking at the wrong ReduceDriver? As you can see in the DriverStrategy enum there is several different ReduceDrivers depending on the strategy the optimizer chooses. best, Markus 2015-04-26 12:26 GMT+02:00 Andra Lungu : > Hey guys, > > I am trying to add a new runtim

Re: Fault Tolerance for Flink Iterations

2015-04-22 Thread Markus Holzemer
ckpoint (by file) and then start the program again with the source that > refers to those files. > > That way, you go through the proper stack (optimizer and jobgraph > generator) that inserts all the necessary partition and > sort operations. > > Greetings, > Stephan > &

Fault Tolerance for Flink Iterations

2015-04-21 Thread Markus Holzemer
Hi everybody, I am writing my master thesis about making flink iterations / iterative flink algorithms fault tolerant. The first approach I implemented is a basic checkpointing, where every N iterations the current state is saved into HDFS. To do this I enabled data sinks inside of iterations, the

[jira] [Created] (FLINK-1885) Bulk mode for gelly

2015-04-14 Thread Markus Holzemer (JIRA)
Markus Holzemer created FLINK-1885: -- Summary: Bulk mode for gelly Key: FLINK-1885 URL: https://issues.apache.org/jira/browse/FLINK-1885 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-1846) Sinks inside of iterations

2015-04-08 Thread Markus Holzemer (JIRA)
Markus Holzemer created FLINK-1846: -- Summary: Sinks inside of iterations Key: FLINK-1846 URL: https://issues.apache.org/jira/browse/FLINK-1846 Project: Flink Issue Type: Improvement