Re: JVM Metaspace for Task Mangers and Job Managers are not getting released.

Hi Ajinkya, The command 'jcmd GC.run' can be used to trigger fullgc for JVM process. However, it should be noted that this may have a performance impact on the ongoing computation. Best, Shammon FY On Mon, May 15, 2023 at 10:51 AM Ajinkya Pathrudkar < ajinkya.pathrudka...@gmail.com> wrote: >

How to pass the TLS certs to the latest version of flink-connector-pulsar

I am trying to connect my Flink application to a Pulsar topic for ingesting data. The topic is active and i am able to ingest the data via a normal Java application. When i try to use the Flink application to ingest the data from the same topic, using the latest version of flink-connector-pulsar i

Re: (无主题)

Hi, It's related to FLINK-11695 which has not been resolved until now. You could increase the limit size of hdfs to alleviate this problem. BTW, You could also share or check something before modifying the configuration: >From the logic of your jo

Re: Issues using PyFlink

Hi Jill, I suspect that the PyFlink isn't installed in the Python environment which is used to run the example. Could you share the complete command you used to execute the example: `./flink-1.17.0/bin/flink run -pyclientexec venv/bin/python3 --python flink-1.17.0/examples/python/ datastream/word_

Re: JVM Metaspace for Task Mangers and Job Managers are not getting released.

Ok, I will check. On Sun, May 14, 2023 at 9:39 PM Shammon FY wrote: > Hi Ajinkya, > > The memory of metaspace may need to be released through FullGC, you can > try to trigger fullgc manually in JobManager and TaskManager, and check > whether the metaspace is released. > > Best, > Shammon FY > >

Re: Flink Job Failure for version 1.16

Hi, I may have missed something, So could you share more: I have recently migrated from 1.13.6 to 1.16.1, I can see there is a > performance degradation... Are you referring to a decrease in Checkpoint Performance when you mention performance decline? It just happens when you upgrade from 1.13.

Query on RestartPipelinedRegionFailoverStrategy

Hi, I am testing the Flink Fine-Grained Recovery from Task Failures on Flink 1.17 and am facing some issues where I need some advice. Have a jobgraph below with 5 operators, and all connections b

(无主题)

Flink 1.10.0 checkpoint to hdfs failed: Caused by:org.apache.flink.util.SerializedThrowable: The directory item limit of .../shared is  Exceeded:limit=1048576 items=1048576

Re: JVM Metaspace for Task Mangers and Job Managers are not getting released.

Hi Ajinkya, The memory of metaspace may need to be released through FullGC, you can try to trigger fullgc manually in JobManager and TaskManager, and check whether the metaspace is released. Best, Shammon FY On Sat, May 13, 2023 at 4:01 PM Jiadong lu wrote: > Hi, Ajinkya > > Maybe some threads

FlinkMl

hello, For my master thesis i am comparing ml frameworks on data streams. What is the current status on FlinkML? Is distributed learning possible on multiple nodes? If yes, how? I played around with FlinkML a bit and modeled a simple pipeline for sentiment analysis on tweets. For this I used the

Re: [Flink K8s Operator] Automatic cleanup of terminated deployments

There is no such feature currently, Kubernetes resources usually do not delete themselves :) The problem I see here is by deleting the resource you lose all information about what happened, you won't know if it failed or completed etc. What is the use-case you are thinking about? If this is someth

[Flink K8s Operator] Automatic cleanup of terminated deployments

Hi all, Currently, if a job turns into terminated status (e.g. FINISHED or FAILED), the flinkdeployment remains until a manual cleanup is performed. I went through the docs but did not find any way to clean them up automatically. Am I missing something? Thanks! Best, Paul Lam