Re: How to scale a Kafka Cluster, what all should we consider

2025-02-03 Thread Brebner, Paul
, Paul From: Sachin Mittal Date: Monday, 3 February 2025 at 3:41 pm To: users@kafka.apache.org Subject: Re: How to scale a Kafka Cluster, what all should we consider [You don't often get email from sjmit...@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentific

Re: How to scale a Kafka Cluster, what all should we consider

2025-02-02 Thread Sachin Mittal
as message rate, message size, fan-out, > number of consumers and partitions etc, all of which potentially consume > CPU – good luck! > > Regards, Paul Brebner > > From: Sachin Mittal > Date: Friday, 31 January 2025 at 7:41 pm > To: users@kafka.apache.org > Subject: How

Re: How to scale a Kafka Cluster, what all should we consider

2025-02-02 Thread Brebner, Paul
potentially consume CPU – good luck! Regards, Paul Brebner From: Sachin Mittal Date: Friday, 31 January 2025 at 7:41 pm To: users@kafka.apache.org Subject: How to scale a Kafka Cluster, what all should we consider [You don't often get email from sjmit...@gmail.com. Learn why this is impo

How to scale a Kafka Cluster, what all should we consider

2025-01-31 Thread Sachin Mittal
Hi, I just wanted to have some general discussions around the topic of how to scale up a Kafka Cluster ? Currently we are running a 5 node Kafka Cluster. Each node has* 4 vcpu *and *8 GiB* memory. I have a topic which is partitioned *35* ways. I have *5* producers publishing messages to that topi