Thanks,
My question is mostly about dynamic resource optimization,
say I configured my application with 30 partitions and then I managed for 30
consumers (within a consumer group) to read/process the produced messages, but
say at for instance at some unpeak load, I realized that a single consu
This depends on the design of your application and how you are using Kafka.
For instance, if you are using Kafka as a message queuing app, using
consumers within a consumer group to load balance, then you should create
the topics with as many partitions as the max number of consumers within
the sa
On 2020/11/13 09:09:00, John Coleman wrote:
> Hi,
>
> We have very high latency ~1 second when we send a small number of small
> messages to Kafka. The whole group of messages is less than 1k. We are
> thinking better to just send the messages together as 1 message containing a
> collectio
Given a business application that resorts into a message queue solution like
Kafka, what is the best number of partitions to select for a given topic? what
influences such a decision?
On the other hand, say we want to achieve a maximal throughput of message
consumption but at minimal resource
