Re: Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized.

Hi Junrui, Thanks for your answer! Since this exception is not very meaningful, is there a solution or a flink config to filter out or silent such exception in batch mode? When I run some large scale batch jobs in a session cluster, it turns out that the JM log will be fulfilled with this except

Re: Uneven TM Distribution of Flink on YARN

Maybe you can check if you have set yarn.taskmanager.node-label for some flink jobs? Best, Biao Geng 发送自 Outlook for iOS 发件人: Chen Zhanghao 发送时间: Tuesday, August 29, 2023 12:14:53 PM 收件人: Lu Niu ; Weihua Hu 抄送: Kenan Kılıçtepe ; user 主题:

Re: Flink CEP Resource Utilisation Optimisation

. Best, Biao 获取 Outlook for iOS<https://aka.ms/o0ukef> 发件人: Abhishek Singla 发送时间: Monday, March 27, 2023 12:38:59 AM 收件人: Geng Biao 抄送: user@flink.apache.org 主题: Re: Flink CEP Resource Utilisation Optimisation Thanks, Geng for the quick and actionable respon

Re: Flink CEP Resource Utilisation Optimisation

Hi Abhishek, Thanks for sharing the experiment! As for the performance question, I believe you could give a try on Flink CEP with version >= 1.16.0, which includes the optimization introduced in FLINK-23890. This optimization will reduce lots

Re: Flink Operator in an off-line k8s enviroment

Hi Mark, I guess you have to create your own local image registry service which your k8s cluster can connect to and upload the image of flink k8s operator to the service. After that, you can run something like `helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator

Re: Does kubernetes operator support manually triggering savepoint with canceling the job?

Hi liting, Maybe you can check codes of deleteClusterDeployment. When savepoint is finished, the operator will delete the job. Is the job not deleted as expected? Best, Bias Geng 获取 Outlook for iOS 发件人: Liting Liu (litiliu) 发送时间: Tuesday,

Re: How to open a Prometheus metrics port on the rest service when using the Kubernetes operator?

Hi Javier, You can use podTemplate to expose the port in the flink containers. Here is a snippet: spec: flinkVersion: v1_15 flinkConfiguration: state.savepoints.dir: file:///flink-data/flink-savepoints state.checkpoints.dir: file:///flink-data/flink-checkpoints metrics.reporter.pro

Re: flink on yarn job always restart

it receive SIGNAL 15 2. is it because of some configuration? (e.g. deploy timeout causing kill?) Replied Message ---- From Geng Biao<mailto:biaoge...@gmail.com> Date 07/18/2022 22:36 To SmileSmile<mailto:a511955...@163.com>、user<mailto:user@flink.apache.org> Cc Subject Re: fli

Re: flink on yarn job always restart

Hi, One possible direction is to check your YARN log or TM log to see if the YARN RM kills the TM for some reason(e.g. physical memory is over limit) and as a result, the JM will try to recover the TM repeatedly according to your restart strategy. The snippet of JM logs you provide is usually n

Re: Kubernetes Operator - Logging

Hi Daniel, I believe it is possible to use logback for jobs submitted by flink kubernetes operator but I have not found an out-of-box solution either. I tried following steps: * Create custom flink image following https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/advanced

Re: Questions regarding classpath loading order in YarnClusterDescriptor

Hi Leon, You are welcome. ‘Each plugin is loaded through its own classloader’(see doc) and as a result, they are not added to the flink system classpath. If I understand correctly, you do not need to do

答复: Application mode deployment through API call

Hi Leon, If you are using application mode through Flink on YARN, the codes in YARNApplicationITCase may be helpful, which has similar function calls with CliFrontend but may be more straightforward. Side notes: in my own experience of using Flink, it is not a very good idea to run a flink jar

答复: Flink Job Execution issue at Yarn

Hi Anitha, If I understand correctly, your JM/TM process memory is larger than the maximum physical memory(i.e. 4m > 32*1024=32768m). So for a normally configured YARN cluster, it should be impossible to launch the Flink JM/TM on worker nodes due to the limit of `yarn.scheduler.maximum-allo

答复: Helm install flink-kubernetes-operator failed

Hi Spoon, The command in current doc (helm install flink-kubernetes-operator helm/flink-kubernetes-operator) should be executed under the repo’s root dir (e.g. ~/flink-kubernetes-operator/). The community are working on to make this process simpler(https://github.com/apache/flink-kubernetes-ope