orpiske commented on PR #14745: URL: https://github.com/apache/camel/pull/14745#issuecomment-2210972636
At first it wasn't very clear to me how the user would be losing messages after the pause. The KafkaConsumer documentation says that [`pause`](https://kafka.apache.org/22/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#pause-java.util.Collection-) _"Suspend fetching from the requested partitions. Future calls to [poll(Duration)](https://kafka.apache.org/22/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#poll-java.time.Duration-) will not return any records from these partitions until they have been resumed using [resume(Collection)](https://kafka.apache.org/22/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#resume-java.util.Collection-)"_. But looking more closely to the reproducer, I think I kinda understand the problem ... It's in part because of a misuse of the API (and, poor documentation for this particular feature). In particular, our documentation states: _"The pausable EIP is meant to be used as a support mechanism when there is an **exception** somewhere in the route that prevents the exchange from being processed. More specifically, the check called by the pausable EIP should be used to test for transient conditions preventing the exchange from being processed."_ Specifically, the trigger for those transient problems should be pulled by an exception, so that [we have a failure in the processing result](https://github.com/apache/camel/blob/main/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/streaming/KafkaRecordStreamingProcessorFacade.java#L97-L117). Otherwise, [we never cause the pause](https://github.com/apache/camel/blob/main/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/errorhandler/KafkaConsumerListener.java#L71-L88), thus leading to them losing the messages. So, to make it short: the reproducer from the user is wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
