Hi everybody,
does Flink offer any automated TTL-functionality for operator state?
My current solution for deleting state after some time is using a
SessionWindow which triggers after a timeout and sends messages to stateful
operators which then call the clear() method on their state.
This is my solution in pseudocode:
sessionEndNotifications =
dataStream.window(EventTimeSessionWindows.withGap(Time.hours(1)).apply(new
StateInvalidatorFunction).keyBy(...);
dataStream.connect(sessionEndNotifications)
.flatMap(new StatefulFlatMapFunction())
.addSink(...);
The StatefulFlatMapFunction calls state.clear() in its flatMap2 function
when it receives a sessionEndNotification. The flatMap1 function contains
the actual business logic which is being calculated per event.
I feel like I'm doing this in a roundabout way. Are there better ways of
doing this?
Best Regards,
Julian
