Thanks for the proposal, I definitely see the need for this improvement, +1.

Regards,
Roman


On Tue, Apr 30, 2024 at 3:11 PM Piotr Nowojski <pnowoj...@apache.org> wrote:

> Hi Yanfei,
>
> Thanks for the feedback!
>
> > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2
> > processes a watermark, the watermark will be sent to downstream, if
> > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when
> > is the watermark sent downstream?
>
> The watermark would be outputted by an operator only once all relevant
> timers are fired.
> In other words, if firing of timers is interrupted a continuation mail to
> continue firing those
> interrupted timers is created. Watermark will be emitted downstream at the
> end of that
> continuation mail.
>
> > 2. IIUC, processing-timer's firing is also encapsulated into mail and
> > executed in mailbox. Is processing-timer allowed to be interrupted?
>
> Yes, both firing processing and even time timers share the same code and
> both will
> support interruptions in the same way. Actually I've renamed the FLIP from
>
> > Interruptible watermarks processing
>
> to:
>
> > Interruptible timers firing
>
> to make this more clear.
>
> Best,
> Piotrek
>
> wt., 30 kwi 2024 o 06:08 Yanfei Lei <fredia...@gmail.com> napisał(a):
>
> > Hi Piotrek,
> >
> > Thanks for this proposal. It looks like it will shorten the checkpoint
> > duration, especially in the case of back pressure. +1 for it!  I'd
> > like to ask some questions to understand your thoughts more precisely.
> >
> > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2
> > processes a watermark, the watermark will be sent to downstream, if
> > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when
> > is the watermark sent downstream?
> > 2. IIUC, processing-timer's firing is also encapsulated into mail and
> > executed in mailbox. Is processing-timer allowed to be interrupted?
> >
> > Best regards,
> > Yanfei
> >
> > Piotr Nowojski <pnowoj...@apache.org> 于2024年4月29日周一 21:57写道:
> >
> > >
> > > Hi all,
> > >
> > > I would like to start a discussion on FLIP-443: Interruptible watermark
> > > processing.
> > >
> > > https://cwiki.apache.org/confluence/x/qgn9EQ
> > >
> > > This proposal tries to make Flink's subtask thread more responsive when
> > > processing watermarks/firing timers, and make those operations
> > > interruptible/break them apart into smaller steps. At the same time,
> the
> > > proposed solution could be potentially adopted in other places in the
> > code
> > > base as well, to solve similar problems with other flatMap-like
> operators
> > > (non windowed joins, aggregations, CepOperator, ...).
> > >
> > > I'm looking forward to your thoughts.
> > >
> > > Best,
> > > Piotrek
> >
>

Reply via email to