Hi,

If you are worried about old state, you can combine the compaction filter based 
TTL with other cleanup strategies (see docs). For example, setting 
`cleanupFullSnapshot` when you take a savepoint it will be cleared of any 
expired state and you can then use it to bring it into Flink 1.8.

Best,
Stefan

> On 13. Mar 2019, at 13:41, Ning Shi <nings...@gmail.com> wrote:
> 
> Just wondering if anyone has any insights into the new TTL state cleanup 
> feature mentioned below.
> 
> Thanks,
> 
> —
> Ning
> 
> On Mar 11, 2019, at 1:15 PM, Ning Shi <nings...@gmail.com 
> <mailto:nings...@gmail.com>> wrote:
> 
>> It's exciting to see TTL state cleanup feature in 1.8. I have a question 
>> regarding the migration of existing TTL state to the newer version.
>> 
>> Looking at the Pull Request [1] that introduced this feature, it seems like 
>> that Flink is leveraging RocksDB's compaction filter to remove stale state. 
>> I assume this means that state will only be cleaned on compaction. If I have 
>> a significant amount of stale TTL state, some of which may have already been 
>> compacted to higher levels already, upgrading to 1.8 may not clean them. Is 
>> this assumption correct? If so, is the best approach to take a full 
>> snapshot/checkpoint and restore it to 1.8 to have them been cleaned on 
>> initialization?
>> 
>> Thanks,
>> 
>> [1] https://github.com/dataArtisans/frocksdb/pull/1 
>> <https://github.com/dataArtisans/frocksdb/pull/1>
>> 
>> --
>> Ning

Reply via email to