On Wed, Sep 6, 2023 at 3:07 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote:
> ... there were no changes to the publishing path recently that may have > directly caused this ... > I wrote "directly", because there was a change that might have influenced it indirectly. Our code used to throw away the Kafka producer and create a new one, every time when hitting any of the more severe exceptions. The assumption is that earlier versions of Kafka client could not recover from some errors, but that's not the case anymore. We removed this code about two months ago and haven't experienced any issues, also during rolling restarts of our Kafka cluster. We think there is a chance of a pre-existing issue in the Kafka producer, which we were not able to see because of closing the client too eagerly. At the same time we see that Kafka streams, for example, does defer to restart the application on certain (un-retriable) errors, which also creates a new producer instance. What is the general guidance here? Are there "fatal" errors where it's better to close the producer and start from scratch? At least we don't see this in the documentation (apart from the OutOfOrderSequenceException, but again — it's not a "must" close, but rather "better" close). Regards, -- Alex
