Hi Edward, if you use partition number and/or partition offset you are
potentially building up a technical debt and tightly coupling the pipeline.
What if you want to change the numbers of partitions in the future? Also
there is what you mentioned about moving to another Kafka cluster where you
hav
Oh, the Kafka parallel consumer may help potentially?
https://www.instaclustr.com/blog/improving-apache-kafka-performance-and-scalability-with-the-parallel-consumer-part-2/
Paul
From: Edvard Fagerholm
Date: Tuesday, 30 May 2023 at 6:55 am
To: users@kafka.apache.org
Subject: Patterns for gener
Hi Edvard, interesting problem – I’ve had similar problems with high fan out
use cases, but only for demo applications where I’m more interested in scale
than order – e.g. have a look at this list of blogs, examples include Anomalia
Machina for Kafka+Cassandra, and Kongo, Kafka+IoT.
https://www
