Hi Banu, Rocksdb is intelligently built to clear any un-useful state from its purview. So you should be good and any required cleanup will be automatically done by RocksDb itself. >From the current documentation, it looks quite hard to relate Flink Internal DS activity to RocksDB DS activity. In my opinion, it is indifferent to the application. This is a good doc that gives insights - https://flink.apache.org/2018/01/30/managing-large-state-in-apache-flink-an-intro-to-incremental-checkpointing/#how-it-works
Regards Saurabh On Mon, Jul 22, 2024 at 10:38 AM banu priya <banuke...@gmail.com> wrote: > Dear Community, > > Gentle reminder about my below email. > > Thanks > Banu > > On Sat, 20 Jul, 2024, 4:37 pm banu priya, <banuke...@gmail.com> wrote: > >> Hi All, >> >> I have a flink job with RMQ Source, filters, tumbling window(uses >> processing time fires every 2s), aggregator, RMQ Sink. >> >> I am trying to understand about states and checkpoints(enabled >> incremental rocksdb checkpoints). >> >> In local rocks db directory, I have .sst files, log, lock, options files. >> >> I read that states are cleared once tumbling window is fired. Does it >> mean my local rocksdb directory's .sst files will be deleted once windows >> fires??. I understand that compaction happens once .sst files reaches (64MB >> it is configurable). Is there any other way .sst files will be/can be >> deleted when tumbling window fires?? >> >> Happy Learning. Happy Weekend. >> >> Thanks >> Banu >> >>
