Hi,
I'm seeing strange things with bitcask expiration: When a new node joins
a Ring, it gets data from the other nodes to rebalance. It seems that
the starting date to count expiration of this data is the time when they
were copied to the new node, *not* the date they were originally created
in the Ring
For instance, on a Ring with 2 weeks expiration:
I write 1 key at T=0 in a Ring with 3 nodes.
At T+7days I add a new node 4. This node gets the ownership of the key.
It seems that this key will expire on node 4 only at T+21 instead of T+14
That's annoying because we have a lot of data that is constantly being
written and that *must* be expired at proper time.
Here is a graph (warning, it shows the free disk space, not the disk
space used) of data expiration when adding new nodes.
https://www.dropbox.com/s/1od5xo5wxsvk11j/render_1.png?dl=0
Is it a wrong configuration ? Here is my bitcask expiration configuration:
multi_backend.bitcask_mult_events.storage_backend = bitcask
multi_backend.bitcask_mult_events.bitcask.data_root =
/var/lib/riak/bitcask_mult_events
multi_backend.bitcask_mult_events.bitcask.expiry = 8d
multi_backend.bitcask_mult_events.bitcask.expiry.grace_time = 1d
multi_backend.bitcask_mult_events.bitcask.io_mode = erlang
multi_backend.bitcask_mult_events.bitcask.max_file_size = 4GB
multi_backend.bitcask_mult_events.bitcask.merge.policy = always
multi_backend.bitcask_mult_events.bitcask.merge.thresholds.dead_bytes = 4GB
multi_backend.bitcask_mult_events.bitcask.merge.thresholds.fragmentation
= 100
multi_backend.bitcask_mult_events.bitcask.merge.triggers.dead_bytes = 4GB
multi_backend.bitcask_mult_events.bitcask.merge.triggers.fragmentation = 100
Maybe the configuration is not right, let me know if you think so. What
I try to do here is to limit disk usage by compacting a file only if it
contains only expired keys. Our data is pure readonly.
Thanks,
dams.
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
