I had a discussion with Tryn Mirell on Riak IRC. Here are a few things discussed
1. LevelDB (till 1.1.x) suffers from compaction stalls 2. A benchmark to illustrate compaction stall ( http://cl.ly/2U3F1h3N2U3L461l000H) 3. Significant work being done in 1.2.x branch on (i) bloom filter support for LevelDB (ii) handling LevelDB compactions 4. More detail on compaction stalls ( https://gist.github.com/fec80f5e1d4971eb506c) 5. Artificially reducing write speeds will only delay the stall, not prevent it. 6. Currently, you will initially see high throughput on LevelDB for a single node, which will eventually drop to around 400 trans/sec. 7. Compaction does not happen on a per bucket basis as data is stored as {bucket:key, value}. It occurs for the entire KV Store. Last point above contradicted our initial assumption the compaction stall is causing the performance degradation because if I pump data into a brand new bucket then I get better write performance. However, the write speed continues to be low on the older large bucket. -- Yousuf Fauzan http://fauzism.com On Sun, Jul 1, 2012 at 4:42 AM, Yousuf Fauzan <yousuffau...@gmail.com>wrote: > Hello, > > Record size ~ 600 bytes, Indexed on 3 fields > > For a new bucket, I am getting around 1-1.5K writes/second. However, when > the bucket size gets large (15 Million records in my case) then the write > speed drops 5-6 times. > > Is this an expected behavior or am I doing something wrong? > > -- > Yousuf Fauzan > http://fauzism.com >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
