Hi, Sometime back i was informed on the group that in production we should never run kafka on same physical machine. So based on that I have a question on how to divide the server nodes we have to run zookeper and kafka brokers.
I have a following setup Data center 1 Lan 1 (3 VMs) 192.168.xx.yy1 192.168.xx.yy2 192.168.xx.yy3 Right now here we are running a cluster of 3 nodejs web servers. These collect data from web and write to kafka queue. Each VM has 70 GB of space. Lan 2 (3 VMs) 192.168.zz.aa1 192.168.zz.aa2 192.168.zz.aa3 These are served the cluster of our database server. Each VM has 400 GB of space. Date center 2 Lan 1 (3 VMs) 192.168.yy.bb1 192.168.yy.bb2 192.168.yy.bb3 Three new machines where we plan to run a cluster of new database to be served as sink of kafka stream applications. Each VM has 400 GB of space. These have connectivity only between Lan 2 of Data center 1 with a 100MBs of data transfer rate. Each VM has a 4 core processor and 16 GB of RAM. They all run linux. Now I would like my topics to be replicated with a factor of 3. Since we don't foresee much volume of data, I don't want it to be partitioned. Also we would like one server to be used as streaming application server, where we can run one or more kafka stream applications to process the topics and write to the new database. So please let me know what is a suitable division to run brokers and zookeeper. Thanks Sachin
