Yes, I was using a changelog. You bring up a good point. I think I need to think harder about what I am trying to do. Maybe deleting all the keys isn't that bad. Especially is I amortise it over the life of the next period.
It seems like waiting for TTLs is probably the right thing to do ultimately. Thanks for the timely response! Ben On Sun Feb 15 2015 at 23:43:27 Chris Riccomini <criccom...@apache.org> wrote: > Hey Benjamin, > > You're right. Currently you have to call .all(), and delete everything. > > RocksDB just committed TTL support for their Java library. This feature > allows data to automatically be expired out. Once RocksDB releases their > TTL patch (I believe in a few weeks, according to Igor), we'll update Samza > 0.9.0. Our tracker patch is here: > > https://issues.apache.org/jira/browse/SAMZA-537 > > > Is there no way to just say I don't care about the old data, gimme a new > store? > > We don't have this feature right now, but we could add it. This feature is > a bit more complicated when a changelog is attached, since we will have to > execute deletes for every key (we still need to call .all()). Are you > running with a changelog? > > Cheers, > Chris > > On Sun, Feb 15, 2015 at 10:41 AM, Benjamin Edwards <edwards.b...@gmail.com > > > wrote: > > > Hi, > > > > I am trialling samza for some windowed stream processing. Typically I > want > > to aggregate a bunch of state over some window of messages, process the > > data, then drop the current state. The only way that I can see to do that > > at the moment is to delete every key. This seems expensive. Is there no > way > > to just say I don't care about the old data, gimme a new store? > > > > Ben > > >
