The physical memory you need depends on the type of workload you are
running and the particular setup for this workload, e.g. number of
partitions, and production and consumption patterns.
Assuming you have 90 MB/s of *production* throughput, a replication
factor of 3 across topics, even repartiti
Hi George,
The 90Mb/s, how did you calculate that?
We are expecting 360 messages (events) per second across all topics we
have. One event size = 256Kb
With regards,
Gowtham S, MCA
On Wed, 19 Feb 2020 at 11:01, George wrote:
> Hi there
>
> with regard to "> Can you please suggest me how to ca
Hi there
with regard to "> Can you please suggest me how to calculate this!"
No I can't thats why I asked, getting idea only way you going to figure
this out is by building 6 servers, with say 12-16 vCPU's assigned and then
run test/volume load and see what Prometheus shows you from a CPU/memory
Thanks, George and Alexandre for the reply.
George, to answer your question
> How did you get to 90MB/s - this is our expected throughput.
For the RAM size and CPU count?
> Can you please suggest me how to calculate this!
With regards,
Gowtham S, MCA
PH: 9597000673
On Wed, 19 Feb 2020 at
Asking as I also need to do some of this math.
How did you get to 90MB/s, the RAM size and CPU count ?
You missing some dedicated SSD's for the ZK's
Assume you doing licensing for 6 brokers...
I'd suggest maybe rather go with 5 x 1TB drives, you provisioning 24TB for
a 23.4 TB sizing, cutting it v
hmm Math here doe snot look right
> > Retention Period = Retention period + (log.retention.check.interval.ms +
> > log.segment.delete.delay.ms ) / 1000
> > = 86400 + (3 + 6000)/1000
> > = 86400 + 360
> > = 86760 Seconds
Hello Gowtham,
You need to include the size of offset and time index files in your
calculations, plus potentially transaction indexes.
If you use default values, that means 10 MB each for every log
segment, which default size is itself 1 GB.
Alexandre
Le lun. 27 janv. 2020 à 08:51, Gowtham S a
Hi,
Please help/guide me to identify the size of the cluster.
Thanks and Regards,
Gowtham S.
On Fri, 24 Jan 2020 at 14:31, Gowtham S wrote:
> Hi,
>
> We are in the process of deploying Kafka in our service. We need to decide
> the machine capacity plan, we arrived at the below formulae for de
Hi,
We are in the process of deploying Kafka in our service. We need to decide
the machine capacity plan, we arrived at the below formulae for deriving
total machine capacity.
Total Broker machine size = Message size per second * Retention period *
Replication Factor
Am I need to consider the
