Re: ZLIB Vulnerability Exposure in Flink statebackend RocksDB

2022-12-08 Thread Vidya Sagar Mula
ksdb/wiki/Compression > [2] https://issues.apache.org/jira/browse/FLINK-30321 > [3] https://cwiki.apache.org/confluence/display/FLINK/1.17+Release > [4] > https://github.com/ververica/frocksdb/blob/FRocksDB-6.20.3/FROCKSDB-RELEASE.md > > -- > Best, > Yanfei > Ververica

ZLIB Vulnerability Exposure in Flink statebackend RocksDB

2022-12-06 Thread Vidya Sagar Mula
Hi, There is a ZLIB vulnerability reported by the official National Vulnerability Database. This vulnerability causes memory corruption while deflating with ZLIB version less than 1.2.12. Here is the link for details... https://nvd.nist.gov/vuln/detail/cve-2018-25032#vulnCurrentDescriptionTitle

Re: Task Manager restart and RocksDB incremental checkpoints issue.

2022-11-14 Thread Vidya Sagar Mula
ity, I would recommend first upgrading to > a later, supported version like Flink 1.16. > > Best regards, > > Martijn > > On Sat, Nov 12, 2022 at 8:07 PM Vidya Sagar Mula > wrote: > >> Hi Yanfei, >> >> Thank you for the response. I have follow up answer and

Re: Task Manager restart and RocksDB incremental checkpoints issue.

2022-11-12 Thread Vidya Sagar Mula
skManager crashes, the `RocksDBKeyedStateBackend` can't > be disposed/closed normally, so the existing rocksdb instance directory > would remain. > > BTW, if you use Application Mode on k8s, if a TaskManager(pod) crashes, > the rocksdb directory would be deleted as the pod is released.

Task Manager restart and RocksDB incremental checkpoints issue.

2022-11-10 Thread Vidya Sagar Mula
Hi, I am using RocksDB state backend for incremental checkpointing with Flink 1.11 version. Question: -- For a given Job ID, Intermediate RocksDB checkpoints are stored under the path defined with "" The files are stored with "_jobID+ radom UUID" prefixed to the location. Case : 1 -

RocksDB checkpoints clean up with TaskManager restart.

2022-11-09 Thread Vidya Sagar Mula
Hi, I am using RocksDB state backend for incremental checkpointing with Flink 1.11 version. Question: -- For a given Job ID, Intermediate RocksDB checkpoints are stored under the path defined with "" The files are stored with "_jobID+ radom UUID" prefixed to the location. Case : 1 -

Re: Incremental checkpointing & RocksDB Serialization

2022-03-07 Thread Vidya Sagar Mula
release-1.13/docs/ops/monitoring/back_pressure/ > [2] > https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/datastream/fault-tolerance/custom_serialization/ > > Best, > Yun Tang > -- > *From:* Vidya Sagar Mula > *Sent:* Sund

Issue with incremental checkpointing size

2021-12-06 Thread Vidya Sagar Mula
Hi, In my project, we are trying to configure the "Incremental checkpointing" with RocksDB in the backend. We are using Flink 1.11 version and RockDB with AWS : S3 backend Issue: -- In my pipeline, my window size is 5 mins and the incremental checkpointing is happening for every 2 mins. I am