Hello, with recent Kafka, would it be okay to have about 1000 topics, with between 1000 to 3000 partitions each, on a 6-node Kafka cluster with replication factor 3?
Each partition would be written to by one producer and consumed by one consumer, with about 2 messages per minute coming in. Would Kafka and its management overhead be okay given the large number of partitions in the millions? Kind Regards, Sven