Hi Tony, before Flink 1.8 expired state is only cleaned up, when you try to access it after expiration, i.e. when user code tries to access the expired state, the state value is cleaned and "null" is returned. There was also already the option to clean up expired state during full snapshots ( https://github.com/apache/flink/pull/6460). With Flink 1.8 expired state is cleaned up continuously in the background regardless of checkpointing or any attempt to access it after expiration.
As a reference the linked JIRA tickets should be a good starting point. Hope this helps. Konstantin On Fri, Mar 8, 2019 at 10:45 AM Tony Wei <tony19920...@gmail.com> wrote: > Hi everyone, > > I read the Flink 1.8 release notes about state [1], and it said > > *Continuous incremental cleanup of old Keyed State with TTL* >> We introduced TTL (time-to-live) for Keyed state in Flink 1.6 (FLINK-9510 >> <https://issues.apache.org/jira/browse/FLINK-9510>). This feature >> allowed to clean up and make inaccessible keyed state entries when >> accessing them. In addition state would now also being cleaned up when >> writing a savepoint/checkpoint. >> Flink 1.8 introduces continous cleanup of old entries for both the >> RocksDB state backend (FLINK-10471 >> <https://issues.apache.org/jira/browse/FLINK-10471>) and the heap state >> backend (FLINK-10473 <https://issues.apache.org/jira/browse/FLINK-10473>). >> This means that old entries (according to the ttl setting) are continously >> being cleanup up. > > > I'm not familiar with TTL's implementation in Flink 1.6 and what new > features introduced in Flink > 1.8. I don't understand what difference between these two release version > after reading the > release notes. Did they change the outcome of TTL feature, or provide new > TTL features, or just > change the behavior of executing TTL mechanism. > > Could you give me more references to learn about it? A simple example > to illustrate it is more > appreciated. Thank you. > > Best, > Tony Wei > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/release-notes/flink-1.8.html#state > -- Konstantin Knauf | Solutions Architect +49 160 91394525 <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Data Artisans GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
