On Thu, May 10, 2012 at 3:01 PM, David Smith <diz...@basho.com> wrote:
> On Thu, May 10, 2012 at 2:33 PM, Tim Haines <tmhai...@gmail.com> wrote:
>
> > I've set up a new cluster, and have been doing pre-deployment benchmarks
> on
> > it. The benchmark I was running slowly sunk from 1000 TPS to 250 TPS over
> > the course of the single 8 hour benchmark doing 1 read+1 update using 1k
> > values. I'm wondering if anyone might have suggestions on how I can
> improve
> > this.
>
> Generally, this suggests that you are becoming seek-time bound. The
> test config, as specified, will generate a pretty huge number of
> not_founds which are (currently) crazy expensive w/ LevelDB,
> particularly as the dataset grows.
>
> Assuming you start with an empty database, a sample of this test will
> generate operations like so:
>
> Key 1000 - get -> not_found
> Key 1001 - update -> not_found + write
> Key 1002 - get -> not_found
> etc..
>
> I.e. the leveldb cache never gets a chance to be useful, because
> you're always writing new values and the cost of writing each new
> value goes up, since you have to thrash the cache to determine if
> you're ever seen the key that doesn't exist. :)
>
> The root problem here is going to be the key_generator --
> partitioned_sequential_int will just run through all the ints in order
> and never revisit a key.
>
>
> > {write_buffer_size, 16777216},
> > {max_open_files, 100},
> > {block_size, 262144},
> > {cache_size, 168430088}
>
> I strongly recommend not changing write_buffer_size; it can have
> unexpected latency side-effects when LevelDB compaction occurs.
> Smaller == more predictable.
>
> Does that help?
>
> D.
>
> --
> Dave Smith
> VP, Engineering
> Basho Technologies, Inc.
> diz...@basho.com
>
Hey David,
With the adjusted ring size and settings, and adjusted to only do puts (so
no missed reads), my cluster is doing about 400 puts per second:
http://twitpic.com/9jnhlm/full
I have yet to insert much data, so as I understand it, as more levels are
created this is going to slow down even further. That still seems pretty
slow for a 5 node cluster.
I also ran a little benchmark on only reads that I knew would hit the
cache, and was achieving about 5000 reads per second.
Cheers,
Tim.
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
