Is incremental checkpointing append-only (forever growing) on s3?

2025-02-18 Thread Vararu, Vadim via user
Hi, I have a large state (currently ~300GB). I’ve enabled the incremental checkpointing and it looks good. However, I don’t think that deletions in the state will reflect in deletions in s3. In this case it will upload more data to s3 when something is added to the state and won’t delete data

Unwanted/processing and resource usage with cross join unnest and multiple sinks

2025-02-18 Thread Teunissen, F.G.J. (Fred) via user
Hello Flink Community, We question on how Flink SQL optimizes the SQL statements and whether this can be influenced. For the sake of simplicity, let’s assume we three small tables, one input and two output tables. The input table contains an array which we want * to flatten. * do some (

Re: Please help with Flink + InfluxDB

2025-02-18 Thread Nikola Milutinovic
Hi Siva. We are running Flink on Kubernetes, deploying via Flink Kubernetes Operator v1.10. And Flink is running in Native K8s mode. For our setup, I add one more deployment: Flink Deployer. It is an image built off the Flink we are using for the Session cluster, holding all the libraries we n

Re: Bug in Flink 'run' for Python Job ?

2025-02-18 Thread Nikola Milutinovic
Hi Siva. I believe, when we built our Flink image (Flink 1.20.0 + several connectors + our Python library code), we had to make a symbolic link: ln -s /usr/bin/python3 /usr/bin/python Try that. Or put a full path in -pyexec argument. Nix. From: Siva Krishna Date: Monday, February 17, 2025 at