Re: Out of order messages when kafka streams application catches up

Tomasz you'll need to upgrade the kafka Streams dependency to 3.0 (or above) to get the fix that John mentioned before -- this behavior is known/expected on earlier versions such as 2.8.1 as you are using On Tue, Nov 15, 2022 at 2:21 AM Tomasz Gac wrote: > Hi John, > > I've reviewed the test you

Re: Cannot send in state COMMITTING_TRANSACTION

Interesting, this does look like it could be a bug in Streams and I'm not aware of any known or already-fixed issues resembling this. Could you file a bug report over here and include as much context/information as possible? Providi

Re: Leader election strategy

Hi Pierre, You may try to use cruise control: https://github.com/linkedin/cruise-control I didn't try it yet but it has task which may help you to auto-balance partitions in the cluster. BR, Mikhail On Tue, Nov 15, 2022 at 3:17 PM sunil chaudhari wrote: > Hi, > Use confluent. It has auto bala

Re: Leader election strategy

Hi, Use confluent. It has auto balancing feature. You dont need to do these manual things. On Tue, 15 Nov 2022 at 7:22 PM, Pierre Coquentin wrote: > Hello Luke, and thank you for your answer. > What I would have hoped for is something more automatic, something that > will spread the load when a

Re: Leader election strategy

Hello Luke, and thank you for your answer. What I would have hoped for is something more automatic, something that will spread the load when a Kafka broker goes down without any human intervention. The reassign script is a bit complicated, you need to generate the topics and partitions list, then g

Re: Out of order messages when kafka streams application catches up

Hi John, I've reviewed the test you sent and it seems to be correct, but it may not reproduce our setup. We are using Java 8, kafka-client 2.8.1 with kafka streams version 2.8.1 against the kafka broker version 2.1.1. We are running it as an OSGI bundle with dependencies packaged within the bundl

Cannot send in state COMMITTING_TRANSACTION

Hi group, We've encountered a problem during regular operation of the kafka-streams application. While processing a record we received the following error. There's very little documentation on this kind of problem but I've gathered that it's a synchronization issue between kafka consumer and produ