Alexander, quick question - do you have Active Anti-Entropy turned on? If
yes, check out this discussion:
http://riak-users.197444.n3.nabble.com/Active-Anti-Entropy-with-Bitcask-Key-Expiry-td4027688.html(might
shed more light on the matter).
> is there a cheap way to figure out the number of keys stored in a bucket?
No. You can do a streaming list keys, which is less expensive than listing
keys used to be, but I'm not sure that counts as cheap.
> how long does it take to purge old keys from the storage?
This depends on how often Bitcask merges are performed. The expired keys
are actually removed from disk when merging/compaction occurs (though until
then, they will return as not found to client requests).
> is it possible to see how many keys were expired for the last
minute/hour/day?
I don't think so - Bitcask doesn't send any notification to Riak when it
expires the keys, and I don't think it keeps any records of its own.
On Wed, Aug 28, 2013 at 4:56 AM, Alexander Ilyin <alexan...@rutarget.ru>wrote:
> Hi,
>
> we're using Riak 1.3.1 with a Bitcask storage engine on a 4 node cluster.
> Properties of the bucket used:
> {"props":{"allow_mult":false,"
> basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":2,"name":"c","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"one","rw":"quorum","small_vclock":50,"w":"one","young_vclock":20}}
>
>
> 3 nodes crashed today morning almost simultaneously, fourth node had
> crashed earlier this nignt. It's hard to figure out the reason of the crash
> looking into the logs, but I suppose this is due to insufficient RAM
> available on the servers (this maillist doesn't like big attachments but I
> can send logs if it will help).
>
> Actually, I'm dealing with Riak memory usage for some time already. Some
> time ago I figured out that memory usage is much more than it is promised
> in the documentation (
> http://riak-users.197444.n3.nabble.com/memory-consumption-td4028674.html).
> After that I tried to reduce number of keys stored, changing the
> expiry_secs setting from 2 months to 1 month. I expected the memory usage
> to drop in half because new keys are put into Riak at a approximately same
> rate for last two months and Bitcask stores all keys in memory. But that
> didn't happen. Memory usage have dropped a little but grew again in a few
> days.
>
> So the questions are:
> * is there a cheap way to figure out the number of keys stored in a bucket?
> * how long does it take to purge old keys from the storage?
> * is it possible to see how many keys were expired for the last
> minute/hour/day?
> * are there some other ways to reduce memory usage I'm missing except for
> adding new servers?
>
> Thanks in advance.
>
> _______________________________________________
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
