Kafka in 2.5 data centers

2025-04-17 Thread Zeratul 1
Hi   Could you please give some clues about the pitfalls of this solution below?   We have 3 ZooKeepers in 3 data centers. 6 Kafka nodes in 2 data centers replication factor=4 insync =3 unclean.leader.election.enable=false We use rack.awareness to distribute partitions across data centers.   We wa

Re: Idempotent producer and handling OutOfOrderSequenceException

2025-04-17 Thread Stig Rohde Døssing
Thanks. What I'm looking for isn't that messages can't be interleaved, it's something like causal order. We're updating a database where every node has their own copy of the database state (similar to RocksDB), by sending changes to that database via Kafka. In our application thread 1 is creating

KTable restoration

2025-04-17 Thread Chad Preisler
In the latest version of Kafka Streams does KTable restoration happen in its own thread? Can you point me to where that happens in the code base? Thanks, Chad

External State Store and EOS still "broken"?

2025-04-17 Thread Chad Preisler
Hello, Way back in 2021 there was a thread on here about using Redis as a custom state store. One of the issues brought up was that EOS was "broken" with custom external state stores. Is that still true with the latest Streams implementation? Thanks, Chad