Hi, David, I would recommend to keep a separate table of closed sessions as a "queue", ordered by the time the session is closed. And in your window method, just create an iterator in the "queue" and only make progress toward the end of the "queue", and do a point deletion in the sessionStore, which I assume that would be keyed by the sessionId.
The reason for that is: 1) RocksDB is a KV-store and it is super efficient in read/write by key, not by iterator 2) If you have to use iterator, making sure that the iterator only goes toward the "tail" where all meaningful work items will be is important to achieve fast and efficient operation. Please refer to this blog from RocksDB team: https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB -Yi On Mon, Jun 6, 2016 at 2:25 PM, David Yu <david...@optimizely.com> wrote: > We use Samza RocksDB to keep track of our user event sessions. The task > periodically calls window() to update all sessions in the store and purge > all closed sessions. > > We do all of this in the same iterator loop. > > Here's how we are doing it: > > > public void window(MessageCollector collector, TaskCoordinator coordinator) > throws Exception { > > KeyValueIterator<String, Session> it = sessionStore.all(); > > while (it.hasNext()) { > > Entry<String, Session> entry = it.next(); > Session session = entry.getValue(); > > update(session); > > if (session.getStatus() == Status.CLOSED) { > sessionStore.delete(entry.getKey()); > } else { > sessionStore.put(entry.getKey(), session); > } > } > } > > > The question is: is this the correct/efficient way to do a read+update for > RocksDB? > > Thanks, > David >
