Having multiple brokers on the same node has a couple of problems for a production installation:
1. You'll have multiple brokers contending for disk and memory resources 2. You could have your partitions replicated to the same node which means if that node fails you would lose data. I think you are better off having 3 nodes with 3 brokers. You can keep with 9 partitions in case you want to add physical nodes in the future and have a replication factor of 2 or 3. -Dave Dave Tauzell | Senior Software Engineer | Surescripts O: 651.855.3042 | www.surescripts.com | dave.tauz...@surescripts.com Connect with us: Twitter I LinkedIn I Facebook I YouTube -----Original Message----- From: Amit K [mailto:amitk....@gmail.com] Sent: Monday, July 18, 2016 8:55 PM To: users@kafka.apache.org Subject: Regarding kafka partition and replication Hi, I have 3 nodes, each with 3 brokers, Kafka cluster along with 3 zookeeper cluster. So total 9 brokers spread across 3 different machines. I am adhered to Kafka 0.9. In order to optimally use the infrastructure for 2 topics (as of now, is not expected to grow drastically in near future), I am thinking of having 9 partitions with 3 (or 6?) replication factor. Will this help me having good distribution of partitions and replicas across brokers? System does not have hugh load (<50 requests/sec of less than 1 kb load each) as of now and is neither expected to get higher load than this. If this replication and partition does not help, please suggest a better topic partition and replication strategy. Also please guide me with any articles or documents about setting up multi node Kafka cluster in regard to partition, replication, general properties to be used (a kind of good practice etc.) Thanks, Amit This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
