Re: Strange behaviour of the flatMap Collector

2016-08-11 Thread Yassin Marzouki
Indeed, using the same parallelism corrected the output. Thank you! On Thu, Aug 11, 2016 at 2:34 PM, Stephan Ewen wrote: > Hi! > > The source runs parallel (n tasks), but the sink has a parallelism of 1. > The sink hence has to merge the parallel streams from the source, which > happens based on

Re: Strange behaviour of the flatMap Collector

2016-08-11 Thread Stephan Ewen
Hi! The source runs parallel (n tasks), but the sink has a parallelism of 1. The sink hence has to merge the parallel streams from the source, which happens based on arrival speed of the streams, i.e., its not deterministic. That's why you see the lines being mixed. Try running source and sink wi

Strange behaviour of the flatMap Collector

2016-08-11 Thread Yassin Marzouki
Hi all, When I use out.collect() twice inside a faltMap, the output is sometimes and randomly skewed. Take this example: final StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment(); env.generateSequence(1, 10) .flatMap((Long t, Collector out) -> {