Incremental checkpoints for Flink

2015-12-19 Thread Marius Melzer
Hello everyone, in order to look for a subject for my diploma thesis, I was at the Flink Forward conference in October. I talked to Aljoscha and some others there and after the Fault Tolerance talk on day 2 I arrived at the idea that an incremental checkpointing of the state of a process when a ba

Re: Incremental checkpoints for Flink

2016-03-03 Thread Marius Melzer
Carbone/tech-talk-google-on-flink-fault-tolerance-and-ha > [5] > https://github.com/apache/flink/blob/55fd5f32d7ef0292a01192ab08456fae49b91791/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java > > > On 19 Dec 2015, at 17:04, Marius Melzer > mailto:mar...@rasumi.n

Understanding Operator Chaining

2016-06-24 Thread Marius Melzer
Hi, I'm currently familiarizing myself with the Flink Code and I have a question regarding operator chaining, maybe somebody could explain this to me? The default Chaining Strategy is HEAD (meaning that the predecessor of the operator will not be chained, thus this operator may be the start of a

forward()

2016-06-26 Thread Marius Melzer
Hi, I was reading a bit of code about partitioning and I have two questions about the forward() method of DataStream: 1) I was before under the impression that forwarding means sending records between operators with same parallelism always from partition 1 to 1, 2 to 2 etc. But this doesn't seem