Re: Aggregation for last n seconds for each event

2020-02-13 Thread Kostas Kloudas
Hi Oleg, With the approach with the MapState you can always fire on every incoming element :) You just iterate in the map state and find all the elements that have timestamp (key) between the timestamp of the current element (NOW) and and NOW-N. Anyway, if Fanbin's solution works, then you can al

Re: Aggregation for last n seconds for each event

2020-02-12 Thread Kostas Kloudas
Hi Oleg, Could you be more specific on what do you mean by "for events of last n seconds(time units in general) for every incoming event."? Do you mean that you have a stream of parallelism 1 and you want for each incoming element to have your function fire with input the event itself and all the

Re: Aggregation for last n seconds for each event

2020-02-11 Thread Fanbin Bu
can u do RANGE BETWEEN INTERVAL '1' SECOND PRECEDING AND CURRENT ROW? On Tue, Feb 11, 2020 at 12:15 PM oleg wrote: > Hi Community, > > I do streaming in event time and I want to preserve ordering and late > events. I have a use case where I need to fire an aggregation function > for events of la

Aggregation for last n seconds for each event

2020-02-11 Thread oleg
Hi Community, I do streaming in event time and I want to preserve ordering and late events. I have a use case where I need to fire an aggregation function for events of last n seconds(time units in general) for every incoming event. It seems to me that windowing is not suitable since it may