Re: Operational concerns with state (was Re: Window limitations on groupBy)

2017-01-19 Thread Raman Gupta
[1] [2]. > > Best, Fabian > > [1] http://data-artisans.com/turning-back-time-savepoints/ > [2] http://data-artisans.com/savepoints-part-2-updating-applications/ > > 2017-01-19 19:02 GMT+01:00 Raman Gupta : >> >> I was able to get it working well with the original approach yo

Operational concerns with state (was Re: Window limitations on groupBy)

2017-01-19 Thread Raman Gupta
dow after a certain period of inactivity to clean up > the state. > > Best, Fabian > > 2017-01-19 1:43 GMT+01:00 Raman Gupta <mailto:rocketra...@gmail.com>>: > > Thank you for your reply. > > If I were to use a keyed stream with a count-based window of 2, &

Re: Window limitations on groupBy

2017-01-18 Thread Raman Gupta
ata at some > point in time. A ProcessFunction can help here, because you can > register a timer which > you can use to evict up old state. > > Hope this helps, > Fabian > > 2017-01-18 15:39 GMT+01:00 Raman Gupta <mailto:rocketra...@gmail.com>>: > > I am inve

Window limitations on groupBy

2017-01-18 Thread Raman Gupta
I am investigating Flink. I am considering a relatively simple use case -- I want to ingest streams of events that are essentially timestamped state changes. These events may look something like: { sourceId: 111, state: OPEN, timestamp: } I want to apply various processing to these state c