Re: [Flink Kubernetes Operator] How to enable Flink history server for the Flink jobs managed by Flink Kubernetes operator

2024-08-20 Thread Gyula Fóra
Hi Alan! The job.id remains the same as the last-state mode uses flinks internal failover mechanism to access the state. We cannot change the job.id while doing this unfortunately. Savepoint upgrades on the other hand would generate a new job id (at least after a recent fix on operator main). I a

[Flink Kubernetes Operator] How to enable Flink history server for the Flink jobs managed by Flink Kubernetes operator

2024-08-20 Thread Alan Zhang
Hi, We are using Apache Flink Kubernetes operator to manage the deployment lifecycle of our Flink jobs. And we are using the application mode with "last-state" upgrade mode for each FlinkDeployment. As I know, each FlinkDeployment will keep using the same job id across different job deployments /

Re: Flink filesystem connector with regex support

2024-08-20 Thread Feng Jin
Glad it worked. At the same time, I think adding filters to the FileSystem Source is a fairly common scenario, but the current documentation lacks relevant examples. We could improve this section with examples and also add relevant methods to the Builder class to facilitate passing the necessary f

Re: Flink filesystem connector with regex support

2024-08-20 Thread amogh joshi
Thanks Feng. Above code worked well and served its purpose. I just modified it to use *NonSplittingRecursiveAllDirEnumerator* instead of *NonSplittingRecursiveEnumerato**r* as the regex filter was to be applied on the individual filenames, excluding the parent directory specified as the path. Fi