Re: Number of kafka topics/partitions supported per cluster of n nodes

2015-07-28 Thread Prabhjot Bharaj
@Jiefu Gong, Are the results of your tests available publicly? Regards, Prabhjot On Tue, Jul 28, 2015 at 10:35 PM, Prabhjot Bharaj wrote: > I would be using the servers available at my place of work. I dont have > access to AWS servers. I would starting off with a small number of nodes in > th

Re: Number of kafka topics/partitions supported per cluster of n nodes

2015-07-28 Thread Prabhjot Bharaj
I would be using the servers available at my place of work. I dont have access to AWS servers. I would starting off with a small number of nodes in the cluster and then plot a graph with x-axis as the number of servers in the cluster and y-axis as the number of topics with partitions, before the cl

Re: Number of kafka topics/partitions supported per cluster of n nodes

2015-07-28 Thread JIEFU GONG
I think these would definitely be useful statistics to have and I've tried to do similar tests! The biggest difference is probably going to be the hardware specs on whatever cluster you decide to run it on. Maybe benchmarks performed on different AWS servers would be helpful too, but I'd like to se

Number of kafka topics/partitions supported per cluster of n nodes

2015-07-27 Thread Prabhjot Bharaj
Hi, I'm looking forward to a benchmark which can explain how many total number of topics and partitions can be created in a cluster of n nodes, given the message size varies between x and y bytes and how does it vary with varying heap sizes and how it affects the system performance. e.g. the resu