Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi, I found some problems with `FunctionWindows` when I implemented this pip, and I added it to PIP: Implementation[4]. After I submit the first PR, you can refer to it. Thanks, Baodi Shi > On Jun 2, 2022, at 18:4232, 石宝迪 wrote: > >> Ok. I would add in the Compatability change another section

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

> It should be reflected in the release notes somehow - don't know the process for that. Yes, we are using the label `release/important-notice` to track the important things we need to highlight in the release not. I have added the label. I support the proposal. Thanks, Penghui On Thu, Jun 2, 2

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

> Ok. I would add in the Compatability change another section with bold or > capital letters to highlight you're creating a breaking change. It should > be reflected in the release notes somehow - don't know the process for that. Ok, I added to `Incompatible case`. PTAL. Thanks, Baodi Shi > 202

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

> > I tend to fail. Although this breaks the current logic. but the current > implementation can be considered is a bug. Ok. I would add in the Compatability change another section with bold or capital letters to highlight you're creating a breaking change. It should be reflected in the release no

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi Baodi, Thanks for the reply and update of the PIP. 1. Pulsar Functions currently isn't integrated with the Transaction feature yet, so there's no EXACTLY_ONCE support. 2. And Yes, "EFFECTIVELY_ONCE = ATLEAST_ONCE + Message Deduplication" On Tue, May 31, 2022 at 9:16 AM 石宝迪 wrote: > >> If

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

>> If you fail to start the function, you immediately break people's > functions when they upgrade to this version. How about notifying them once > via logger (WARN)? I tend to fail. Although this breaks the current logic. but the current implementation can be considered is a bug. > It will flo

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Regarding > > >1. I would add: Validate existing ProcessingGuarantee test for >AtMostOnce, AtLeastOnce, ExactlyOnce still works (when autoAck=true) > > Remove "I would add:" On Tue, May 31, 2022 at 12:24 PM Baozi wrote: > Hi, Asaf. > > Thanks review. > > >> I'm not entirely sure that is

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi Baodi, Regarding > >1. When the delivery semantic is ATMOST_ONCE, add verify autoAck must >be true. If the validation fails, let the function fail to start (This >temporarily resolves the configuration ambiguity). When autoAck is >subsequently removed, the message will be acked

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

>> This I don't understand fully. When you introduce MANUAL > ProcessingGuarantee configuration, isn't configuration universal to all > languages? When someone uses pulsar-admin to create a function, regardless > of the language, they can specify Processing Guarantee. If we're adding > this ability

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Only commiters can modify wikis, we can discuss that first. Thanks, Baodi Shi > 2022年5月30日 22:5828，Asaf Mesika 写道： > > Still new to the PIP process: Shouldn't this PIP be added here > ? > > On Thu, May 12, 2022 at 11:52 AM Ba

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi, Asaf. Thanks review. >> I'm not entirely sure that is accurate. The Effectively-Once as I > understand it is achieved using transactions, thus the consumption of that > message and production of any messages, as a result, are considered one > atomic unit - either message acknowledged and mes

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Still new to the PIP process: Shouldn't this PIP be added here ? On Thu, May 12, 2022 at 11:52 AM Baozi wrote: > Hi Pulsar community, > > I open a https://github.com/apache/pulsar/issues/15560 for Function add > NONE delivery se

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

On Mon, May 30, 2022 at 4:24 PM Baozi wrote: > Hi, Rui. > > Thanks review. > > > 1. API changes should also contain the changes of `Function.proto`, > including new `ProcessingGuarantees` option and `autoAck`. > > > I added to pip. > > > 2. Please be sure the other language runtimes (like Python,

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Thanks for applying the fixes. 1. Regarding > >- EFFECTIVELY_ONCE: The message is acknowledged *after* the function >finished execution. Depends on pulsar deduplication, and provides >end-to-end effectively once processing. > > I'm not entirely sure that is accurate. The Effectively-O

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi, Rui. Thanks review. > 1. API changes should also contain the changes of `Function.proto`, including > new `ProcessingGuarantees` option and `autoAck`. I added to pip. > 2. Please be sure the other language runtimes (like Python, Golang) do > support similar `record.ack()` function from t

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi, Neng. Thank review. > 1. Instead of deleting the `autoAck`, keep it but not use it in the code. > And documented clearly it's deprecated for the following 2~3 release. And > then delete it. Great! I changed PIP. Refer to the latest pip. https://github.com/apache/pulsar/issues/15560 > 2. Fo

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi, Mesika. Thanks review. >> 2. I suggest calling it `MANUAL` `ProcessingGuarantee` instead of NONE. As >> you carefully explained, ProcessingGuarantee comes does to the fact that >> the function executor calls acknowledge, in specific timing: Added, Refer to the latest pip. https://github.com

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Hi Baodi, Nice work. Put some suggestions below, ptal. 1. API changes should also contain the changes of `Function.proto`, including new `ProcessingGuarantees` option and `autoAck`. 2. Please be sure the other language runtimes (like Python, Golang) do support similar `record.ack()` function fr

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

1. "Added NONE delivery semantics and delete autoAck config." - Added --> add 2. I suggest calling it `MANUAL` `ProcessingGuarantee` instead of NONE. As you carefully explained, ProcessingGuarantee comes does to the fact that the function executor calls acknowledge, in specific timing: - `AT_MO

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Some suggestions: 1. Instead of deleting the `autoAck`, keep it but not use it in the code. And documented clearly it's deprecated for the following 2~3 release. And then delete it. 2. For `PulsarSinkAtLeastOnceProcessor` and `PulsarSinkEffectivelyOnceProcessor`, if `NONE` is configured, it defaul

[DISCUSS] PIP-166: Function add NONE delivery semantics

Hi Pulsar community, I open a https://github.com/apache/pulsar/issues/15560 for Function add NONE delivery semantics Let me know what you think. Thanks, Baodi Shi ## Motivation Currently Function supports three delivery semantics, and also provides autoAck to control whether to automatical