Excellent Thank you. On Wed, Jul 14, 2021 at 5:53 AM Igal Shilman <i...@ververica.com> wrote:
> Hi Ammon, > > The duration is per item, and the cleanup happens transparently and > incrementally via RocksDB (background compactions with a custom filter) [1] > > In your example a gets cleaned up, while b will be cleaned in ~10min. > > Kind regards, > Igal. > > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/fault-tolerance/state/#cleanup-during-rocksdb-compaction > > On Wed, Jul 14, 2021 at 5:09 AM Ammon Diether <adiet...@gmail.com> wrote: > >> >> Thank you for asking. I meant PersistedTable >> -> >> https://github.com/apache/flink-statefun/blob/release-3.0/statefun-sdk-embedded/src/main/java/org/apache/flink/statefun/sdk/state/PersistedTable.java >> >> It is related to state backend. I am using rocksdb backend. >> >> >> >> On Tue, Jul 13, 2021 at 8:53 PM Caizhi Weng <tsreape...@gmail.com> wrote: >> >>> Hi >>> >>> By PersistentTable do you mean state backend? If yes, the answer differs >>> with different operators and state backends. >>> >>> For keyed states the duration is for per key. However the exact time to >>> clean up a key really depends on the operator and the state backend. Most >>> operators will register a timer and when the timer triggers it cleans up >>> that key. >>> >>> Ammon Diether <adiet...@gmail.com> 于2021年7月14日周三 上午5:17写道： >>> >>>> >>>> Question >>>> If the duration is 20 minutes, >>>> 1) is the duration per item? >>>> 2) or is the duration for the table as a whole? >>>> >>>> Suppose the following items >>>> ("a", "a-value") 30 minutes ago >>>> ("b", "b-value") 10 minutes ago >>>> >>>> Does "a" get cleaned up? or neither gets cleaned up yet because the >>>> most recent item is not above the duration? >>>> >>>> Another question >>>> Also, if enough time passes, does the cleanup happen during a >>>> checkpoint, savepoint, or the next time the PersistentTable is accessed? >>>> If the table is never accessed again, will it remain in the savepoints >>>> forever? >>>> >>>>
