I think these would definitely be useful statistics to have and I've tried to do similar tests! The biggest difference is probably going to be the hardware specs on whatever cluster you decide to run it on. Maybe benchmarks performed on different AWS servers would be helpful too, but I'd like to see these!
On Mon, Jul 27, 2015 at 10:51 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > I'm looking forward to a benchmark which can explain how many total number > of topics and partitions can be created in a cluster of n nodes, given the > message size varies between x and y bytes and how does it vary with varying > heap sizes and how it affects the system performance. > > e.g. the result should look like: t topics with p partitions each can be > supported in a cluster of n nodes with a heap size of h MB, before the > cluster sees things like JVM crashes or high mem usage or system slowdown > etc. > > I think such benchmarks must exist so that we can make better decisions on > ops side > If these details dont exist, I'll be doing this test myself on varying the > values of parameters described above. I would be happy to share the numbers > with the community > > Thanks, > prabcs > -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu <elise...@berkeley.edu> | (925) 400-3427