Re: Jobs running in flink are not viable in YARN resource manager WEB UI

I see. So the screenshots are using version 1.9.3. What does version 1.10.1 show? Best, Weihua On Wed, Mar 29, 2023 at 12:15 PM Kiran Kumar Kathe < kirankumarkathe...@gmail.com> wrote: > I am using yarn-cluster mode and command as ./flink run -m yarn-cluster > /home/kirankathe/s3checkup/origina

Re: Unable to Use spec.flinkVersion v1_17 with Flink Operator

I think you forgot to upgrade the CRD during the upgrade process on your cluster. As you can see here: https://github.com/apache/flink-kubernetes-operator/blob/release-1.4/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml#L38-L44 The newer version already contains suppor

Re: Jobs running in flink are not viable in YARN resource manager WEB UI

Hi, > jobs are visible in YARN resource manager Web UI Which mode are you using to submit a job to YARN? Session mode or Perjob mode. IIUC, the YARN UI only shows Flink clusters, not jobs. Instead, the jobs will be shown in Flink ui. Best, Weihua On Tue, Mar 28, 2023 at 11:16 PM Martijn Visse

Re: python udf out of memory

Hi tom Flink will create individual classloader for each job in task manager. When jobs are frequently started and stopped, the usage of memory in Metaspace will increase. I found out that your OOM was caused by metaspace. I think you can check the size of metaspace and try to increase the size by

Re: Custom source

Thanks Lasse, I think you can create an issue and update the document if there is any wrong Best, Shammon FY On Wed, Mar 29, 2023 at 3:48 AM Lasse Nedergaard < lassenedergaardfl...@gmail.com> wrote: > Hi. > I have figured it out. The documentation are wrong in both places. > > Med venlig hilsen

Unable to Use spec.flinkVersion v1_17 with Flink Operator

Hello Flink Users: I'm trying to upgrade to use Flink 1.17.0 with my pipeline in order to have support for writing to Azure Data Lake Storage. However when I change the `spec.flinkVersion` to v1_17 I get an error message: ```bash The FlinkDeployment "test-replay-streaming-run" is invalid: spec.

Savepoint state keeps growing

Hello! I'm trying to do a savepoint and reconfigure my window aggregation jobs overtime and I noticed that the windowed results keep accumulating with time (I'm using heap backend). I modified the snapshot code and noticed that Flink does not drop my aggregated window states after the window has c

python udf out of memory

Hi, I am running a standalone cluster setup and submit flinksql job with python udf following the examples here github.com/ververica/flink-sql-cookbook/blob/main/udfs/01_python_udfs/01_python_udfs.md

Re: Jobs running in flink are not viable in YARN resource manager WEB UI

Hi, I don't see anything in the 1.10 release notes [1] that would indicate that anything has changed between 1.9 and 1.10 that would indicate that something has changed on this topic. It should work. Best regards, Martijn [1] https://nightlies.apache.org/flink/flink-docs-release-1.10/release-no

Re: Jobs running in flink are not viable in YARN resource manager WEB UI

Actually we are using 1.10.1 in YARN cluster in production , so to make changes in cluster it is bit complicated and it need to check all the jobs from first and need to test everything , so is there any chance of running the with the flink 1.10.1 by doing any configuration changes , that make j

Re: [External] Re: Way to add columns with defaults to the existing table and recover from the savepoint

You could consider trying out the experimental version upgrade that was introduced as part of FLIP-190: https://cwiki.apache.org/confluence/x/KZBnCw On Tue, Mar 21, 2023 at 12:11 PM Ashish Khatkar via user < user@flink.apache.org> wrote: > Hi Shammon, > > Schema evolution works with avro type sta

Re: Flink 1.17 upgrade issue when using azure storage account for checkpoints/savepoints

Hi, This is tracked under https://issues.apache.org/jira/browse/FLINK-31612 and a fix has been merged and will be made available when the first patch version for Flink 1.16.1 will be released. Best regards, Martijn On Sat, Mar 25, 2023 at 9:37 AM ChangZhuo Chen (陳昌倬) wrote: > On Sat, Mar 25,

Re: Jobs running in flink are not viable in YARN resource manager WEB UI

Hi, You can't mix and match different versions; they all need to be the same version. Flink 1.9.3 is no longer supported by the Flink community, I would recommend upgrading to a still supported version (which is currently Flink 1.16 and Flink 1.17) Best regards, Martijn On Tue, Mar 28, 2023 at

Jobs running in flink are not viable in YARN resource manager WEB UI

When I submit jobs to flink using version 1.9.3 those jobs are visible in YARN resource manager Web UI , but when I use flink version 1.10.1 jobs running in flink are not visible in the Web UI , So in order to check where the issue is raising , I started replacing flink 1.10.1 folders with flink

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

Congratulations! Best, Guowei On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan wrote: > Congratulations！ > > Best, > Yuxin > > > Guanghui Zhang 于2023年3月28日周二 11:06写道： > >> Congratulations！ >> >> Best, >> Zhang Guanghui >> >> Hang Ruan 于2023年3月28日周二 10:29写道： >> >> > Congratulations! >> > >> > Best,