Hi Pulsar community,
I open a https://github.com/apache/pulsar/issues/15560 for Function add NONE
delivery semantics
Let me know what you think.
Thanks,
Baodi Shi
## Motivation
Currently Function supports three delivery semantics, and also provides autoAck
to control whether to automatically ack.
Because autoAck affects the delivery semantics of Function, it can be confusing
for users to understand the relationship between these two parameters.
For example, when the user configures `Guarantees == ATMOST_ONCE` and `autoAck
== false`, then the framework will not help the user to ack messages, and the
processing semantics may become `ATLEAST_ONCE`.
The delivery semantics provided by Function should be clear. When the user sets
the guarantees, the framework should ensure point-to-point semantic processing
and cannot be affected by other parameters.
## Goal
Added `NONE` delivery semantics and delete `autoAck` config.
The original intention of `autoAck` semantics is that users want to control the
timing of ack by themselves. When autoAck == false, the processing semantics
provided by the framework should be invalid. Then we can add `NONE` processing
semantics to replace the autoAck == false scenario.
When the user configuration `ProcessingGuarantees == NONE`, the framework does
not help the user to do any ack operations, and the ack is left to the user to
handle. In other cases, the framework guarantees processing semantics.
## API Changes
1. Add `NONE` type to ProcessingGuarantees
``` java
public enum ProcessingGuarantees {
ATLEAST_ONCE,
ATMOST_ONCE,
EFFECTIVELY_ONCE,
NONE
}
```
2. Delete autoAck config in FunctionConfig
``` java
public class FunctionConfig {
- private Boolean autoAck;
}
```
## Implementation
1. In `PulsarSinkAtLeastOnceProcessor` and
`PulsarSinkEffectivelyOnceProcessor`, when `ProcessingGuarantees != NONE` can
be ack.
<https://github.com/apache/pulsar/blob/c49a977de4b0b525ec80e5070bc90eddcc7cddad/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/sink/PulsarSink.java#L274-L276>
2. When the delivery semantic is `ATMOST_ONCE`, the message will be acked
immediately after receiving the message, no longer affected by the autoAck
configuration.
https://github.com/apache/pulsar/blob/c49a977de4b0b525ec80e5070bc90eddcc7cddad/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/JavaInstanceRunnable.java#L271-L276
3. When user call `record.ack()` in function, just `ProcessingGuarantees ==
NONE` can be work.
## Plan test
The main test and assert is that when ProcessingGuarantees == NONE, the
function framework will not do any ack operations for the user.
## Compatibility
1. This change will invalidate the user's setting of autoAck, which should be
explained in the documentation and provide parameter verification to remind the
user.
2. Runtimes of other languages need to maintain consistent processing logic
(python, go).
