Re: Understanding Operator Chaining

2016-06-28 Thread Maximilian Michels
Hi Marius, The current chaining code assumes there will never be too complex chains. Which is probably true for most jobs but could become in issue in some cases. If you want to make sure the chain is broken, you can start a new chain using `startNewChain()` on all single output operators. Cheer

Re: Understanding Operator Chaining

2016-06-27 Thread Aljoscha Krettek
Hi Marius, the chaining code is still somewhat fragile and some stuff in there are leftovers. For example, StreamGroupedFold can only be used on a KeyedStream, which means that it can never be within a chain because the shuffle always breaks a chain. Specifying HEAD here is therefore redundant. I t

Understanding Operator Chaining

2016-06-24 Thread Marius Melzer
Hi, I'm currently familiarizing myself with the Flink Code and I have a question regarding operator chaining, maybe somebody could explain this to me? The default Chaining Strategy is HEAD (meaning that the predecessor of the operator will not be chained, thus this operator may be the start of a