Hi, there,
We will deploy a large-scale kafka cluster using kafka 2.0.0 in production
environment recently, as JunRao recommend in cluster-limit lately, to
accommodate for the rare event of a hard failure of the controller, it is
better to limit each broker to have up to 4,000 partitions and each cluster to
have up to 200,000 partitions. We plan to possess 2k topics each with 4k
partitions and two-replicas, it is clearly reaching the cluster-wide limit
recommended in above blog. Can such a large-scale kafka cluster meet our
production demand well?
ps: broker hardware configuration described as follows:
24 cores
256G mem
10Gbps nic
22*4T sata disk
Any advise/guidance would be greatly appreciated!
Thanks!
