On Thu, May 10, 2012 at 2:33 PM, Tim Haines <tmhai...@gmail.com> wrote:
> I've set up a new cluster, and have been doing pre-deployment benchmarks on
> it. The benchmark I was running slowly sunk from 1000 TPS to 250 TPS over
> the course of the single 8 hour benchmark doing 1 read+1 update using 1k
> values. I'm wondering if anyone might have suggestions on how I can improve
> this.
Generally, this suggests that you are becoming seek-time bound. The
test config, as specified, will generate a pretty huge number of
not_founds which are (currently) crazy expensive w/ LevelDB,
particularly as the dataset grows.
Assuming you start with an empty database, a sample of this test will
generate operations like so:
Key 1000 - get -> not_found
Key 1001 - update -> not_found + write
Key 1002 - get -> not_found
etc..
I.e. the leveldb cache never gets a chance to be useful, because
you're always writing new values and the cost of writing each new
value goes up, since you have to thrash the cache to determine if
you're ever seen the key that doesn't exist. :)
The root problem here is going to be the key_generator --
partitioned_sequential_int will just run through all the ints in order
and never revisit a key.
> {write_buffer_size, 16777216},
> {max_open_files, 100},
> {block_size, 262144},
> {cache_size, 168430088}
I strongly recommend not changing write_buffer_size; it can have
unexpected latency side-effects when LevelDB compaction occurs.
Smaller == more predictable.
Does that help?
D.
--
Dave Smith
VP, Engineering
Basho Technologies, Inc.
diz...@basho.com
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
