Thanks for the proposal, I definitely see the need for this improvement, +1.
Regards, Roman On Tue, Apr 30, 2024 at 3:11 PM Piotr Nowojski <pnowoj...@apache.org> wrote: > Hi Yanfei, > > Thanks for the feedback! > > > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2 > > processes a watermark, the watermark will be sent to downstream, if > > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when > > is the watermark sent downstream? > > The watermark would be outputted by an operator only once all relevant > timers are fired. > In other words, if firing of timers is interrupted a continuation mail to > continue firing those > interrupted timers is created. Watermark will be emitted downstream at the > end of that > continuation mail. > > > 2. IIUC, processing-timer's firing is also encapsulated into mail and > > executed in mailbox. Is processing-timer allowed to be interrupted? > > Yes, both firing processing and even time timers share the same code and > both will > support interruptions in the same way. Actually I've renamed the FLIP from > > > Interruptible watermarks processing > > to: > > > Interruptible timers firing > > to make this more clear. > > Best, > Piotrek > > wt., 30 kwi 2024 o 06:08 Yanfei Lei <fredia...@gmail.com> napisał(a): > > > Hi Piotrek, > > > > Thanks for this proposal. It looks like it will shorten the checkpoint > > duration, especially in the case of back pressure. +1 for it! I'd > > like to ask some questions to understand your thoughts more precisely. > > > > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2 > > processes a watermark, the watermark will be sent to downstream, if > > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when > > is the watermark sent downstream? > > 2. IIUC, processing-timer's firing is also encapsulated into mail and > > executed in mailbox. Is processing-timer allowed to be interrupted? > > > > Best regards, > > Yanfei > > > > Piotr Nowojski <pnowoj...@apache.org> 于2024年4月29日周一 21:57写道： > > > > > > > > Hi all, > > > > > > I would like to start a discussion on FLIP-443: Interruptible watermark > > > processing. > > > > > > https://cwiki.apache.org/confluence/x/qgn9EQ > > > > > > This proposal tries to make Flink's subtask thread more responsive when > > > processing watermarks/firing timers, and make those operations > > > interruptible/break them apart into smaller steps. At the same time, > the > > > proposed solution could be potentially adopted in other places in the > > code > > > base as well, to solve similar problems with other flatMap-like > operators > > > (non windowed joins, aggregations, CepOperator, ...). > > > > > > I'm looking forward to your thoughts. > > > > > > Best, > > > Piotrek > > >
