Dear Flink Community, I’m currently working on deploying a FlinkSessionJob using PyFlink and have encountered an issue I hope someone can help clarify. Here’s my setup:
* I installed the Flink Kubernetes Operator in namespace A * I deployed a FlinkDeployment in namespace B * I’m trying to deploy a FlinkSessionJob in namespace B The job fails with a file not found error related to the jarURI. I’m referencing a JAR file located at /flink-tmp/, which is mounted as a PVC on the FlinkDeployment. This PVC is backed by EFS, so it should be accessible across pods. I’ve confirmed that the file exists at that path. I also tried using the following public JAR instead: jarURI: https://repo1.maven.org/maven2/org/apache/flink/flink-python/1.20.1/flink-python-1.20.1.jar However, even with this change, the Python script located in the same mounted volume (/flink-tmp/transform_confirmed_trade.py) still cannot be found. I haven’t been able to find any best practices or working examples of FlinkSessionJob with PyFlink. Do you have any sample manifests or guidance for this use case? Here is the manifest I’m using: apiVersion: flink.apache.org/v1beta1 kind: FlinkSessionJob metadata: name: transform-confirmed-trade-job namespace: namespace-b spec: deploymentName: flink-cluster job: jarURI: file:///flink-tmp/flink-python-1.20.1.jar entryClass: "org.apache.flink.client.python.PythonDriver" args: ["-pyclientexec", "/usr/bin/python3", "--python", "/flink-tmp/transform_confirmed_trade.py"] parallelism: 2 upgradeMode: stateless Additional Question: Is it possible to configure TaskManagers to run permanently (i.e., not shut down when idle)? If so, what configuration options are required to achieve this behavior? Any help or pointers would be greatly appreciated. Best regards, Pachara Aryuyuen (Kas) Cloud Platform Engineer
