Hey,
so I'm currently benchmarking riak on a bare metal cluster (2xquadcore
xeon 2.2GHz, 24GB RAM, 120GB SSD, gigabit ethernet between nodes)
I wrote some custom benchmarks in basho bench which connected to all
three nodes and I filled up the cluster consisting of 3 nodes with about
68GB of Data (50mio objects in one bucket and 35mio objects in another).
Storage backend is leveldb and caching is tuned according to the wiki guide.
Now with those three nodes i could get up to a constant 30k ops/s - see
http://smyck.org/fast.png for example.
I ran basho bench with a concurrency of 80 workers from another node
next to the three riak nodes. Empirical tests proved that my benchmarks
peaked at 80 workers in terms of ops/s. Above that it just wasn't
getting faster and latency increased.
Then I wanted to add another node to see the cluster scale up in
throughput. What I saw with the same settings except for adding th 4th
node to the basho bench config was this:
http://smyck.org/slow.png
So while it kicked off a lot faster, the performance decreased quickly.
I even increased the leveldb cache further but the result was the same.
Then I reduced the concurrency to 70 and I got a constant 35k ops/s
again. See http://smyck.org/35concurrent.png
Now I did kind of a binary search for the sweet spot and it turned out i
could set the concurrency setting up to 74 - everything above 74 showed
the behavior from http://smyck.org/slow.png
Now - any ideas on that? Is running basho bench from one node the
problem? The machine running basho bench is at around 45% cpu load.
Should it be running like tsung - from multiple nodes?
Any ideas? As I'm fairly new to riak it is also quite likely that I
missed something - so I'd be happy if you'd point me to common mistakes
in scenarios like these.
~ John
--
John-Paul Bader | Software Development
www.wooga.com
wooga GmbH | Saarbruecker Str. 38 | D-10405 Berlin
Sitz der Gesellschaft: Berlin; HRB 117846 B
Registergericht Berlin-Charlottenburg
Geschaeftsfuehrung: Jens Begemann, Philipp Moeser
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
