hi, we have a similar use case, and from my experience it is not simple (if at all) to implement your logic when using a session window. Eventually we used the state + timers approach and we have full control of the cleanup and merge logic.
Thanks Sigalit On Mon, May 12, 2025 at 1:35 PM Sachin Mittal <sjmit...@gmail.com> wrote: > Second approach is good to try out. > I am also solving for a similar problem using this approach only. > > Thanks > Sachin > > > On Mon, 12 May 2025 at 3:52 PM, Ehud Lev <ehud....@forter.com> wrote: > >> Hi Flink community, >> >> We're building a Flink topology that aggregates events by key. When a key >> is seen for the first time, we load its base state from an external store. >> After performing some calculations, we emit the result to Kafka, and >> another process is responsible for writing it back to the store. >> >> Our goals are: >> >> 1. >> >> *Keep per-key state alive for a few hours*, but *remove it if no new >> events arrive during that time*. >> 2. >> >> *Suppress frequent updates* and *emit output only every 5 seconds*, >> but *only if updates occurred* during that interval. >> >> I see two possible implementation strategies: >> >> 1. >> >> *Session Windows (processing time):* >> This would allow automatic state cleanup based on inactivity, but I'm >> concerned about *merging session states*, since our logic depends on >> having a well-defined "base state" and applying updates to it. I can not >> handle merge basically. >> 2. >> >> *KeyedProcessFunction with timers:* >> In this approach, we’d maintain keyed state, and use: >> - >> >> A *5-second periodic timer* to check for and emit updates (if any). >> - >> >> That same timer can also track *last-seen timestamps* to expire >> state after a few idle hours. >> >> *My questions:* >> Is the session window approach even suitable here, given the merging >> issue I have? >> Is the KeyedProcessFunction with a "single recurring timer" a >> recommended pattern in such use cases? >> Are there any caveats or better alternatives for managing "suppressed + >> expiring" keyed state? >> >> >> Thanks in advance! >> >> -- >> Ehud Lev, Staff Engineer >> email: ehud....@forter.com web: www.forter.com >> mobile: 052-5832253 >> >
