RE: Two operators consuming from same stream

2018-01-26 Thread Newport, Billy
We've seen the same thing here. We read files twice for the same reason, it's simply faster to do that than to connect the two pipes to the same input. From: Sofer, Tovi [mailto:tovi.so...@citi.com] Sent: Monday, January 01, 2018 8:51 AM To: user@flink.apache.org Subject: Two operators consuming

RE: Two operators consuming from same stream

2018-01-04 Thread Sofer, Tovi
d”,“ orderId”) .flatMap(mapperB); From: Timo Walther [mailto:twal...@apache.org] Sent: יום ד 03 ינואר 2018 11:02 To: user@flink.apache.org Subject: Re: Two operators consuming from same stream Hi Tovi, I think your code without duplication performs two separate shuffle operations whereas the ot

Re: Two operators consuming from same stream

2018-01-03 Thread Timo Walther
Hi Tovi, I think your code without duplication performs two separate shuffle operations whereas the other code only performs one shuffle. Further latency impacts might be due to the overhead involved in maintaining the partitioning for a keyed stream/key groups and switching key contexts in