Re: Correct semantics of producer close

>> Michael > > >> > > >> > > >> On Wed, Sep 29, 2021 at 7:04 AM Enrico Olivelli > wrote: > > >>> > > >>> I agree that we must ensure that every pending callback must be > completed > > >>> eventually (timeout or another error is not a pro

Re: Correct semantics of producer close

the application can perform a flush() explicitly and also > >>> wait for every callback to be executed if that is the requirement. > >>> > >>> Usually you call close() when: > >>> 1. you have a serious problem: you already know that there is a hard > >>> error, > >

Re: Correct semantics of producer close

have to support case 1: >>> fail fast and close (no need for flush()) . >>> >>> In my experience trying to implement "graceful stops" adds only complexity >>> and false hopes to the users. >>> >>> Enrico >>> >>> &g

Re: Correct semantics of producer close

.com.invalid> > > ha scritto: > > > > > I agree to try to ensure ”at most once“ when closing。 > > > > > > > > > > That would still get controlled by send timeout, after that, the send > > > will fail and close should proceed. > > > Thi

Re: Correct semantics of producer close

ost once“ when closing。 > > > > > > > That would still get controlled by send timeout, after that, the send > > will fail and close should proceed. > > This sounds more in line with “at most once”。 > > > > > > -- 原始邮件 ---------- >

Re: Correct semantics of producer close

his sounds more in line with “at most once”。 > > > -- 原始邮件 -- > 发件人: > "dev" > < > matteo.me...@gmail.com>; > 发送时间

?????? Correct semantics of producer close

I agree to try to ensure ??at most once?? when closing?? > That would still get controlled by send timeout, after that, the send will fail and close should proceed. This sounds more in line with ??at most once --  -- ??:

Re: Correct semantics of producer close

> but equally they might be > surprised when closeAsync doesn't complete because the pending messages > can't be cleared That would still get controlled by send timeout, after that, the send will fail and close should proceed. -- Matteo Merli On Wed, Sep 29, 2021 at 12:52 AM Jack Vanlightly wr

Re: Correct semantics of producer close

I can see both sides of the argument regarding whether to flush pending messages or not. But I think what is definitely in the contract is not to discard any callbacks causing user code to block forever. No matter what, we must always call the callbacks. Personally, I am in favour of a close opera

Re: Correct semantics of producer close

>I don't think that implementing `closeAsync` with graceful shutdown logic implies a guarantee of message publishing. Rather, it guarantees that failures will be the result of a real exception or a timeout. I think that's beside the point. There is no definition of "real" exceptions. At that

Re: Correct semantics of producer close

It’s a good point that `ProducerImpl#failPendingBatchMessages` treats messages in batch container also as pending messages. I agree with your definition of "graceful close”. It’s more like a “at most once” semantics, like the original JavaDoc said > pending writes will not be retried Thanks, Yu

Re: Correct semantics of producer close

Thanks for bringing this thread to the mailing list, Yunze. I think the right change is to update the `closeAsync` method to first flush `batchMessageContainer` and to then asynchronously wait for the `pendingMessages` queue to drain. We could add a new timeout or rely on the already implemented `

Re: Correct semantics of producer close

I can’t agree more, just like what I’ve said in PR 12195: > At any case, when you choose `sendAsync`, you should always make use of the > returned future to confirm the result of all messages. In Kafka, it's the > send callback. But I found many users are confused about the current behavior, es

Re: Correct semantics of producer close

Clients should not depend on any of this behaviour, since the broker is at the other end of an unreliable network connection. The semantic differences are kind of meaningless from a usability point, since flushing on close =/= published. What exactly does "graceful" convey here? Flush the buffe

Correct semantics of producer close

Hi all, Recently I found a PR (https://github.com/apache/pulsar/pull/12195 ) that modifies the existing semantics of producer close. There're already some communications in this PR, but I think it's better to start a discussion here to let more know.