Ok Matthew, We will proceed with the deletion , will monitor the disk space and will come back with further reports to the list.
Thanks for your time, Antonio 2015-07-14 18:32 GMT+01:00 Matthew Von-Maszewski <matth...@basho.com>: > Antonio, > > A Riak delete operation happens in these steps: > > - Riak writes a “tombstone” value for the key to the N vnodes that contain > it (this is a new record) > > - Riak by default, waits 3 seconds to verify all vnodes agree to the > tombstone/delete > > - Riak issues an actual delete operation against the key to leveldb > > - leveldb creates its own tombstone > > - the leveldb tombstone “floats” through level-0 and level-1 as part of > normal compactions > > - upon reaching level-2, leveldb will initiate immediate compaction and > propagation of tombstones in .sst table files containing 1000 or more > tombstones. This is when disk space recovery begins. > > > Yes, this means that initially the leveldb vnodes will grow in size until > enough stuff (new data and/or tombstones) forces the tombstones to level-2 > via normal compaction operations. “Enough stuff” to fill levels 0 and 1 is > about 4.2Gbytes of compressed Riak objects. > > The “get” operation you mentioned is something that happens internally. A > manual “get” by your code will not influence the operation. > > Matthew > > > On Jul 14, 2015, at 1:01 PM, Antonio Teixeira <eagle.anto...@gmail.com> > wrote: > > Hi Matthew, > > We will be removing close to 1 TB of data from the node , and since we are > short on "disk space" when we saw that the disk space was actually rising > we halted the data removal. > > Now according to some docs I have read, if after a deletion ( a few > seconds ) we make a .get() it force the release of the diskspace , is this > true ? > > For us it's not possible to move data to another node, is there any way > even manually to release the space ? or at least to force : > " The actual release occurs significantly later (days, weeks, or even > months later) when the tombstone record merges with the actual data in a > background compaction." > > Thanks for all the help, > Antonio > > > 2015-07-14 17:43 GMT+01:00 Matthew Von-Maszewski <matth...@basho.com>: > >> Antonio, >> >> Here is a detailed discussion of the Riak / leveldb delete scenario: >> >> https://github.com/basho/leveldb/wiki/mv-aggressive-delete >> >> Pay close attention to the section titled “Update April 6, 2014”. This >> explains why as much as 4.2G bytes per vnode might remain within leveldb >> after deleting all keys. There is no mechanism to override the logic that >> causes the disk space retention. >> >> One workaround is to use Riak’s handoff mechanism to transfer vnodes from >> one physical server to another. The vnode transfer will remove all >> deletion tombstones on the destination. The last step of the transfer then >> deletes all leveldb files on the original server, recovering the space. >> >> Matthew >> >> >> >> > On Jul 14, 2015, at 12:32 PM, Antonio Teixeira <eagle.anto...@gmail.com> >> wrote: >> > >> > Hello, >> > >> > We have been migrating our Riak Database to another infrastructure >> through a "streaming" process and right now we should have somewhere around >> 2Gb of free space the Hard Disk, however those 2Gb are still being used by >> Riak. After some research I believe the problem is the Objects are only >> being marked for deletion and not actually deleted at runtime. What we need >> is a way to aggressively deleted those keys or some way to force Riak to >> delete those marked keys and subsequently release the Disk Space. The Riak >> version we are using is v2.0.2 >> > _______________________________________________ >> > riak-users mailing list >> > riak-users@lists.basho.com >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
