[
https://issues.apache.org/jira/browse/FLINK-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727145#comment-16727145
]
Yangze Guo commented on FLINK-10473:
------------------------------------
Hi, [~azagrebin] . I now understand it will work well when set
`NeverReturnExpired` in `StateTtlConfig` and act same with the timer
implementation.
Above is some improvement we can do if we use TTL as a trade-off between
accuracy and capacity. But now it seems like another effort. I will discuss
these in another thread when incremental cleanup merged.
Thanks for the explanation by [~hequn8128] and [~azagrebin] again.
> State TTL incremental cleanup using Heap backend key iterator
> -------------------------------------------------------------
>
> Key: FLINK-10473
> URL: https://issues.apache.org/jira/browse/FLINK-10473
> Project: Flink
> Issue Type: New Feature
> Components: State Backends, Checkpointing
> Affects Versions: 1.7.0
> Reporter: Andrey Zagrebin
> Assignee: Andrey Zagrebin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.8.0
>
>
> This feature enables lazy background cleanup of state with time-to-live in
> state keyed backend which stores state in JVM heap. The idea is to keep a
> global state lazy iterator with loose consistency. Every time a state value
> for some key is accessed or a record is processed, the iterator is advanced,
> TTL of iterated state entries is checked and the expired entries are cleaned
> up. When the iterator reaches the end of state storage it just starts over.
> This way the state with TTL is regularly cleaned up to prevent ever growing
> memory consumption. The caveat of this cleanup strategy is that if state is
> not accessed or no records are processed then accumulated expired state still
> occupies the storage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
