OutOfMemory Failure on Savepoint

2021-06-30 Thread Abhishek SP
Hello, I am observing a failure whenever I trigger a savepoint on my Flink Application which otherwise runs without issues The app is deployed via AWS KDA(Kubernetes) with 256 KPU(6 Task managers with 43 slots each. 1 KPU = 1 vCPU, 4GB Memory, and 50GB Diskspace. It uses RocksDB backend) The sav

S3 Source support in Flink

2021-10-28 Thread Abhishek SP
Hello, I see S3 supported as a Sink through StreamingFileSink but do not see a source equivalent StreamingFileSource *Questions:* 1. What is the current recommendation for using S3 as a continuous

Flink 1.11 Flink 1.13 savepoint compatiblity

2022-01-19 Thread Abhishek SP
Hello, Are Flink 1.11 savepoints compatible with Flink 1.13? We are trying to migrate a production Flink 1.11 app to Flink 1.13. The app uses RocksDB as the statebackend. This documentation

Re: Flink 1.11 Flink 1.13 savepoint compatiblity

2022-01-19 Thread Abhishek SP
Upon searching more, I found the Compatibility Table <https://nightlies.apache.org/flink/flink-docs-master/docs/ops/upgrading/#compatibility-table> which does mention that Flink 1.11 savepoint is compatible with Flink 1.13 On Wed, Jan 19, 2022 at 1:34 PM Abhishek SP wrote: > Hello

Flink Checkpoint Behavior/Missing Records

2023-10-12 Thread Abhishek SP
I have implemented a Custom File Source(wrapper around native Flink File Source) that signalsNoSplits using a custom strategy which terminates the Flink App automatically. Due to external restrictions, I am bound to using Flink 1.13.3 in Streaming mode with Unbounded Input and cannot use the Batch