Got it. Thanks for the explanation. LGTM
Penghui On Thu, Mar 9, 2023 at 3:06 AM Neng Lu <nl...@apache.org> wrote: > Hi everyone, > > This discussion has been one week old. If there's no objection or > concerns, I'll move forward and close the discussion with the conclusion > that we are good with the proposed change. > > This will result in the PR merge (although it was already merged, so the > merged change won't be reverted in this case). > > On 2023/03/07 03:45:02 Neng Lu wrote: > > Hi Penghui, > > > > Thanks for your question. > > > > One case is failure recovery for a windowing function. > > > > A windowing function will ack message until its window is emitted. If > the window function fails due to issues such as OOM and restarts, it has a > massive backlog to catch up. And the function will never be able to recover > itself since the backlog keeps growing and it keeps OOM. > > > > Our user prefers an automatic way for recovery, given they are okay with > skipping some backlog data. (This is acceptable in IoT cases). Also, Users > may deploy hundreds of functions in their environment. Manually resetting > the cursor is not scalable and is a heavy burden for the on-call person in > such cases. > > > > Hope the above use case can help provide some more context regarding the > change. > > > > On 2023/03/03 03:51:35 PengHui Li wrote: > > > Hi Neng, > > > > > > Thanks for raising up the discussion > > > > > > > In certain failure cases, the function needs to skip all the content > > > between the last successfully acked message and the latest message in > the > > > topic in order to skip the huge backlog and quick recovery. > > > > > > Do you have some real cases that can help us to understand it > > > is necessary to introduce a new flag? Another possibility is users > > > can use pulsar admin to reset the cursor to the latest position, > > > Why will it not work for users? > > > > > > Regards, > > > Penghui > > > > > > > On Mar 1, 2023, at 10:16, Neng Lu <nl...@apache.org> wrote: > > > > > > > > In certain failure cases, the function needs to skip all the content > > > > between the last successfully acked message and the latest message > in the > > > > topic in order to skip the huge backlog and quick recovery. > > > > > > > > >
