Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-19 Thread Haiting Jiang
Hi Nitin, Michael's point about the decoupling of producer and consumer makes sense. Your use case is more like a queue model, not a pub-sub model. So the decoupling is not the issue for you. But Pulsar is mainly a pub-sub system by nature. So we must take it into more consideration. As in your

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-16 Thread Asaf Mesika
>From my knowledge, when a client does negative ack, after a certain amount of time, it sends the following command to the broker containing the negatively acked messages ids: message CommandRedeliverUnacknowledgedMessages { required uint64 consumer_id = 1; repeated MessageIdData message_i

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-16 Thread Nitin Goyal
Hello all, I have a few questions. 1. When a message is NACKed does it still send the commands to the broker? 2. Do we have any plan to replace the ReconsumeLater function with NACK Backoff interface in future? On Tue, Jan 10, 2023 at 7:17 PM Nitin Goyal wrote: > Hello Team, > > Thanks for po

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-10 Thread Nitin Goyal
Hello Team, Thanks for pointing these things out. First thing is to consider that this addon feature is for the message which needs to go to DLQ after a retryLater reaches the maximum limit of consumer level retries. It allows an additional maximum limit if a message has one at the time of produc

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-10 Thread r...@apache.org
Thanks for submitting this PIP, Nitin Goyal. Seeing that you have submitted a related pull request in the Go SDK community, I am sorry that I made the request changes first. For the retry processing of a single message, the methods currently provided are: -ReconsumeLater -Nack In actual usage s

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-10 Thread Enrico Olivelli
I think that Michael's point is very important. Producers and Consumers are decoupled and this PIP would introduce a new concept. Also the same Message can be consumed using multiple subscriptions (typically Applications) and all the applications will process the message in a different way (by def

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-09 Thread Michael Marshall
Thanks for submitting this PIP, Nitin Goyal. At the heart of this PIP is an assumption about the relationship between producers and consumers. The PIP assumes a producer knows how many times a consumer should attempt to consume a message before giving up and sending it to the DLQ. Does anyone have

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-09 Thread Nitin Goyal
Hello Enrico, For your concern about temporarily increasing of retry. It can be achieved using overriding msg property while calling reconsuming later with custom properties.. About msg immutable as per current design if consumer call reconsumeLater function it creates a new msg in the system add

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-08 Thread Enrico Olivelli
I don't think that this is a good idea. Because IIUC we want to add a property per message that sets the maximum time of retries. Unfortunately in a real system sometimes you have to change the number of retries temporarily, for instance in case of major system failure you don't want to lose all

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-06 Thread Zike Yang
Hi, This looks good to me. +1 I was thinking if we could add a new API for `reconsumeLater` to let users set the max retry times easily. But I saw that there are too many reconsumeLater API and this will make the consumer more complex. Thanks, Zike Yang On Thu, Jan 5, 2023 at 3:58 PM Zixuan Li

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-04 Thread Zixuan Liu
+1 Thanks, Zixuan Nitin Goyal 于2023年1月5日周四 13:50写道: > Hi all, > > I made a PIP to discuss: https://github.com/apache/pulsar/issues/19136 > > Thanks, > Nitin Goyal >