Re: Manipulating Processing elements of Network Buffers

2018-02-20 Thread m@xi
Hi Till! Thanks a lot for your useful reply. So now I get it. I should not manipulate or disturb the network buffer contents, as this will trigger other problematic behaviours. On the other hand, the price of buffering the data in my operator first and e.g. sorting them first based on some crite

Re: Manipulating Processing elements of Network Buffers

2018-02-19 Thread Till Rohrmann
Hi Max, the network buffer order is quite important at the moment, because the network stream does not only transport data but also control events such as the checkpoint barriers. In order to guarantee that you don't lose data in case of a failure it is (at the moment) strictly necessary that chec

Manipulating Processing elements of Network Buffers

2018-02-15 Thread m@xi
Hello Flinker! I know that one should set appropriately the number of Network Buffers (NB) that its Flink deployment will use. Except from that, I am wondering if one might change/manipulate the specific sequence of data records into the NB in order to optimize the performance of its application.