Hi, How would you evaluate such a query? I think the answer could be that you have to keep all that older data around so that you can evaluate when a new event arrives. In Flink, you could use a ProcessFunction for that and use a MapState that keeps events bucketed into one-week intervals. This way, can more efficiently iterate over the buckets that are required when evaluating a given event and you can also efficiently delete a complete bucket of older events once you know that they are not required anymore.
These are the relevant sections of the Flink doc: - https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/state.html <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/state.html> - https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html> Best, Aljoscha > On 13. Jun 2017, at 15:27, shashank agarwal <shashank...@gmail.com> wrote: > > Hi, > > I have to process each event with last 1 hour , 1 week and 1 month data. Like > how many times same ip occurred in last 1 month corresponding to that event. \ > > I think window is for fixed time i can't calculate with last 1 hour > corresponding to current event. > > If you have any clue please guide what should i use Table, ProcessFunction or > global window. Or what approach should i take ? > > -- > Thanks Regards > > SHASHANK AGARWAL > --- Trying to mobilize the things.... >
