Hi All, I'm really new to Kafka and wanted to know if anyone can help me better understand partition count in relation to the Kafka cluster (apologies in advance for noob questions).
I was requested to increase a topic's partition count from 30 to 100 in order to increase workers' parallelism (there are already other topics in this cluster with 100-200 partition counts per topic). The cluster is built of 4 physical servers. Each server has 132 GB RAM, 40 CPU cores, 6 SAS disks 1.1 TB each. Is PartitionCount:100 considered a high number of partitions per topic in relation to the cluster? Is there a good way for me to predetermine what an optimal partition count might be? Thanks a lot!
