Hi, Thanks for the proposal. Could you provide more background/explanation/motivation why do you need such feature? What do you mean by “network I/O” degradation?
On it’s own burst writes shouldn’t cause problems within Flink. If they do, we might want to fix the original underlying problem and if they are causing problems in external systems, we also might think about other approaches to fix/handle the problem (write rate limiting?), which might be more general and not fixing only bursts originating from WindowOperator. I’m not saying that your proposal is bad or anything, but I would just like to have more context :) Piotrek. > On 26 Sep 2018, at 19:21, Rong Rong <walter...@gmail.com> wrote: > > Hi Dev, > > I was wondering if there's any previous discussion regarding how to handle > burst network I/O when deploying Flink applications with window operators. > > We've recently see some significant network I/O degradation when trying to > use sliding window to perform rolling aggregations. The pattern is very > periodic: output connections get no traffic for a period of time until a > burst at window boundaries (in our case every 5 minutes). > > We have drafted a doc > <https://docs.google.com/document/d/1fEhbcRgxxX8zFYD_iMBG1DCbHmTcTRfRQFXelPhMFiY/edit?usp=sharing> > on > how we proposed to handle it to smooth the output traffic spikes. Please > kindly take a look, any comments and suggestions are highly appreciated. > > -- > Rong
