On Thu, Sep 27, 2012 at 12:36:27PM +0200, Nico Meyer wrote: > If the final fix is contained in the most recent version on github, > then all it effectively does is to add an offset to the expiration > time (for the purpose of triggering merges, not actual expiration).
Yeah, I finally got around to looking through the code. I think this patch will help in that automatic merges will happen, just not every 3 minutes. Since the trigger ends up being checking for data points older than the expiration time + grace period, but the actual merging will remove those older than the expiration time. So this essentially makes the constant merging move from every 3 minutes to every grace period seconds (with the caveat that other triggers still apply, so file size and dead bytes). > The old behaviour can be a problem if you never update or delete any > keys explicitly, which is mostly the case if one records time > series. There is/was no background thread that scans for expired > keys memory, so bitcask does not know how many keys are expired at > any given moment. Background scanning would indeed be one solution. > But maybe one that is too expensive CPU wise. > > After sleeping over the problem, I believe indeed the easiest > solution would be to look at the file creation time, and only > trigger merges for files that are at least an hour old or so (could > be configurable). But in your and my case this will most likely just > degenerate to merging every hour anyway, which is still much better > than merging every 3 minutes, as is the case now. > > Come to think of it, increasing the interval between merge checks > (make it configurable!) might just work as well for most people. So > maybe that's the quickest solution. I think this is essentially what their solution does, they just call it grace time, but it basically changes the interval. We're going to be trying the patch out next week, I'll let you all know how it goes. -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro <antho...@alumni.caltech.edu> _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
