Hi Nic. I do not have a solution (sorrrry), but have seen something similar. And have complained about it, already. Look up my mail in the archives.
I am also deploying our session cluster using Flink Kubernetes operator. I was setting “execution.checkpointing.interval: 300000”. This should set a global value for periodic checkpoints. They do work in Docker Compose. I can see the value is read and accepted in Job Manager, expect the same in Task Manager – haven’t checked the logs. However, all running jobs are stating that “Periodic Checkpointing Disabled”. If I alter the job to set periodic checkpointing on its execution environment, then it works. So, it seems like JM or TM is not passing certain options to our jobs. Mind you, all other checkpointing options were passed into the execution environment of the job. Is this a bug or a feature? Nix, From: Nic Townsend <nictowns...@uk.ibm.com> Date: Friday, January 10, 2025 at 4:22 PM To: user@flink.apache.org <user@flink.apache.org> Subject: table.exec.source.idle-timeout support Hi, I’m deploying Flink 1.19 via the k8s operator. I’m setting `table.exec.source.idle-timeout: 30 s` in the `spec.flinkConfiguration` section of the FlinkDeployment CR. The Flink UI is showing the JobManager has been configured with the value and the JM and TM logs both show `INFO [] - Loading configuration property: table.exec.source.idle-timeout, 30 s`. However, when I run a simple SQL job (select * from <kafkatable>) – where the Kafka topic has 3 partitions, the watermark does not appear to update when a partition is idle (confirmed using the job graph UI). If I instead use the option `'scan.watermark.idle-timeout'=30 s ` in the CREATE TABLE then the watermark does increase when the partition goes idle. I can’t find an error in the JobManager to suggest the config option is invalid or malformed, and I’m not seeing a Jira issue that reflects this. So I feel like I must be doing something wrong, but would appreciate any advice, thank you! -- Nic Townsend IBM Event Processing Senior Engineer / Technical Lead Slack: @nictownsend Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: Building C, IBM Hursley Office, Hursley Park Road, Winchester, Hampshire SO21 2JN