blankensteiner commented on PR #97: URL: https://github.com/apache/pulsar-dotpulsar/pull/97#issuecomment-1123272762
Hi @jonclare Ah, I see. Sadly, like so many other features of Pulsar, negative acknowledgment, multi-topic subscription, RetryLetterTopic, and DeadLetterTopic are not first-class citizens in Pulsar, meaning that the protocol (and to a large extent also the broker) is not aware of those concepts. So instead of handling the complexity server-side, every client has to become complex (and also with the consequence of added load on the network and servers) and get the expected behavior spot on. This is a problem because the documentation on the protocol/behavior level is close to non-existent and the user documentation is lacking the depth needed to act as an implementation guide. So, we have to read the Java Client code and/or reverse engineer the protocol to see how things should be implemented, which is a very time-consuming task. Instead of just copying the Java client, let's have a talk about the wanted feature and see how that could fit into DotPulsar. Generally, I would prefer to build a feature on top of DotPulsar instead of into DotPulsar, if possible. A consumer has the 'Receive' method to get a single message and we have two extension methods that use this method. The 'Messages' extension method will return an IAsyncEnumerable and the consumer also has the 'Process' extension method which takes a Func to call when a message is received. The latter also supports creating traces and will auto-acknowledge the messages. I think this should be the default way to process messages using DotPulsar and Receive/Messages the more low-level interfaces. I was planning to expand 'Process' so that the user could give some options for a degree of parallelism and other useful stuff and think the retry/dead-letter stuff could be implemented here as well? The current implementation will catch exceptions from the user's callback Func and add the exception info to the activity and then acknowledge the message. We could inject an exception handler that could send the message to a retry topic before acknowledging it. I guess 'Process' will need to see if there is a retry topic and then create a consumer for it and of course also create producers for the retry topic and the dead letter topic. Could that be a way to go? /db -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
