@Jiefu Gong, Are the results of your tests available publicly?
Regards, Prabhjot On Tue, Jul 28, 2015 at 10:35 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > I would be using the servers available at my place of work. I dont have > access to AWS servers. I would starting off with a small number of nodes in > the cluster and then plot a graph with x-axis as the number of servers in > the cluster and y-axis as the number of topics with partitions, before the > cluster gives up. > > I need 1 help here: What parameters should I keep in mind for tuning the > JVM, if I have to see best performance ? > My machine specs: I have 4 core CPUs with 8GB RAM with 500GB HDD (RAID > Striped) > > Regards, > Prabhjot > > On Tue, Jul 28, 2015 at 10:27 PM, JIEFU GONG <jg...@berkeley.edu> wrote: > >> I think these would definitely be useful statistics to have and I've tried >> to do similar tests! The biggest difference is probably going to be the >> hardware specs on whatever cluster you decide to run it on. Maybe >> benchmarks performed on different AWS servers would be helpful too, but >> I'd >> like to see these! >> >> On Mon, Jul 27, 2015 at 10:51 PM, Prabhjot Bharaj <prabhbha...@gmail.com> >> wrote: >> >> > Hi, >> > >> > I'm looking forward to a benchmark which can explain how many total >> number >> > of topics and partitions can be created in a cluster of n nodes, given >> the >> > message size varies between x and y bytes and how does it vary with >> varying >> > heap sizes and how it affects the system performance. >> > >> > e.g. the result should look like: t topics with p partitions each can be >> > supported in a cluster of n nodes with a heap size of h MB, before the >> > cluster sees things like JVM crashes or high mem usage or system >> slowdown >> > etc. >> > >> > I think such benchmarks must exist so that we can make better decisions >> on >> > ops side >> > If these details dont exist, I'll be doing this test myself on varying >> the >> > values of parameters described above. I would be happy to share the >> numbers >> > with the community >> > >> > Thanks, >> > prabcs >> > >> >> >> >> -- >> >> Jiefu Gong >> University of California, Berkeley | Class of 2017 >> B.A Computer Science | College of Letters and Sciences >> >> jg...@berkeley.edu <elise...@berkeley.edu> | (925) 400-3427 >> > > > > -- > --------------------------------------------------------- > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- --------------------------------------------------------- "There are only 10 types of people in the world: Those who understand binary, and those who don't"
