Re: Change Streaming Source Function Interface

2015-04-30 Thread Gyula Fóra
Okay, sounds very reasonable :) On Thu, Apr 30, 2015 at 10:15 PM, Stephan Ewen wrote: > For the variant with the "run()" method, this requires strong assumptions > about the internal behavior of the source. > > Unless I am overlooking something, the source needs to guarantee this: > > - It need

Re: Change Streaming Source Function Interface

2015-04-30 Thread Stephan Ewen
For the variant with the "run()" method, this requires strong assumptions about the internal behavior of the source. Unless I am overlooking something, the source needs to guarantee this: - It needs to lock internally and perform the state update and record emit call inside the locked scope -

Re: Change Streaming Source Function Interface

2015-04-30 Thread Gyula Fóra
Hi, The only thing we need is to guarantee that the source will not output any records or update the state while we take the snapshot and send the barrier. There are multiple ways of doing this I guess. We could simply lock on these objects for instance or add the methods you wrote. If we lock, we