On Mon, Nov 7, 2022 at 8:05 PM Matthias J. Sax wrote:
> However, I don't think that there is any guarantee that you might "see"
> concurrent modification (IIRC, RocksDB uses snapshot isolation for
> iterators). But maybe that's good enough for you?
Yes, thankfully that is exactly what I'm after,
Good point about the docs... I guess it must support concurrency do to
IQ, which might iterator over the store while `process()` modifies it. I
was only reasoning about `process()` and punctuation and forgot IQ.
So it seems we indeed have this contract -- what is good new for you.
However, I d
"Matthias J. Sax" writes:
> In general, it's not safe to keep the iterator open, because when process() is
> executed in-between two punctuator calls, it might modify the store and
> invalidate the iterator. There is no guarantee that the returned iterator
> supports concurrency.
This makes sens
In general, it's not safe to keep the iterator open, because when
process() is executed in-between two punctuator calls, it might modify
the store and invalidate the iterator. There is no guarantee that the
returned iterator supports concurrency.
Hence, even if it happens that the currently us
Hello Matthias, thanks for the response!
"Matthias J. Sax" writes:
> Spanning your own thread and calling context.forward() is _not_ safe, and
> there
> is currently no way for you to make is safe. The runtime code makes certain
> assumptions about being single threaded which would break if yo
Your observation is correct. The Processor#process() and punctuation
callback are executed on a single thread. It's by design to avoid the
issue of concurrency (writing thread safe code is hard and we want to
avoid putting this burden onto the user). There is currently no plans to
make process(
I have data that I am storing in a state store which I would like to
periodically run some code over, and the way I have decided to do this is
via a punctuator from inside a Transformer, which gets an iterator over the
state store, performs actions, and forwards events on.
So far, everything works