Re: Issues while upgrading from 1.12.1 to 1.14.0

Yes Nico. I have evaluated this. I have tried below: 1. Take the savepoint 2. Stop the job 3. Shutdown the instances 4. Started new pod using below command: /docker-entrypoint.sh "standalone-job" "-Ds3.access-key=${AWS_ACCESS_KEY_ID} " "-Ds3.secret-key=${AWS_SECRET_ACCESS_KEY}" "-Ds

Re: asyhcnrouonous io question

Hi Tom, On Mon, Oct 4, 2021 at 10:42 PM tom yang wrote: > Hello, > > > > I have a recently ran into an issue with RichAsyncFunction and wanted to > get some guidance from the community > > > > Please see snippet > > > > *class* AsyncFetchFromHttp *extends* RichAsyncFunction String, String>> { >

Re: New session mode job manager deployment rejected existing task managers

Actually we figured it out. We need to configure High Availability mode to recover jobs during new kubernetes deployment. On Tue, Oct 5, 2021 at 11:39 AM Sharon Xie wrote: > Hi, > > I'm currently running Flink 1.13.2 using kubernetes session mode - native > kubernetes. When I update the job mana

Re: RocksDB: Spike in Memory Usage Post Restart

i was reading a bit about RocksDb and it seems the Java version is somewhat particular about how it should be cleaned up to ensure all resources are cleaned up: ttps://github.com/facebook/rocksdb/wiki/RocksJava-Basics#me

New session mode job manager deployment rejected existing task managers

Hi, I'm currently running Flink 1.13.2 using kubernetes session mode - native kubernetes. When I update the job manager deployment through `kubectl apply flink-jobmanager-deployment.yaml`, a new job manager pod is created. I'd expect all the task manager pods will re-register with the new JM pod.

Re: Issues while upgrading from 1.12.1 to 1.14.0

Hi Parag, I am not so familiar with the setup you are using, but did you check out [1]? Maybe the parameter [--fromSavepoint /path/to/savepoint [--allowNonRestoredState]] is what you are looking for? Best regards, Nico [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/re

Re: k8s not able to submit job from jobmanager

I think , I got the answer ( application mode cluster doesn't support running job using cli ) On Tuesday, October 5, 2021, 08:38:24 AM EDT, Israel Ekpo wrote: Your Flink versions are different  Your Docker container has version 1.13.2 but it seems your job is attempting to submit with

Re: Pyflik job data stream to table conversion declareManagedMemory exception

Hi Kamil, On Tue, Oct 5, 2021 at 9:03 AM Kamil ty wrote: > Hello, > > I'm trying to run a pyflink job in cluster mode (with yarn). My job > contains source and sink definitions using Table API which are converted to > a datastream and back. Unfortunately I'm getting an unusual exception at: > *t

Snapshot method for custom keyed state checkpointing ?

Hello, Is there any method available in a RichFunction to be called by Flink with a keyed context each time a checkpoint is triggered please ? It seems that the CheckpointedFunction interface provides such a feature (snapshotState method) but only in case of operator state and it is called in a n

Re: k8s not able to submit job from jobmanager

Your Flink versions are different Your Docker container has version 1.13.2 but it seems your job is attempting to submit with 1.14 in application mode That is the first obvious observation On Tue, Oct 5, 2021 at 5:35 AM Dhiru wrote: > *My DockerFile * > > FROM flink:1.13.2-scala_2.12-java11 >

Issues while upgrading from 1.12.1 to 1.14.0

Hello, We are currently using Apache flink 1.12.0 deployed on k8s cluster of 1.18 with zk for HA. Due to certain vulnerabilities in container related with few jar(like netty-*, meso), we are forced to upgrade. While upgrading flink to 1.14.0, faced NPE, https://issues.apache.org/jira/browse/FLINK

k8s not able to submit job from jobmanager

My DockerFile  FROM flink:1.13.2-scala_2.12-java11 RUN mkdir -p /opt/flink/plugins/flink-s3-fs-hadoopRUN ln -fs /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/flink-s3-fs-hadoop/. RUN mkdir -p /opt/flink/plugins/flink-s3-fs-prestoRUN ln -fs /opt/flink/opt/flink-s3-fs-presto-*.jar /opt

Pyflik job data stream to table conversion declareManagedMemory exception

Hello, I'm trying to run a pyflink job in cluster mode (with yarn). My job contains source and sink definitions using Table API which are converted to a datastream and back. Unfortunately I'm getting an unusual exception at: *table = t_env.from_data_stream(ds, 'user_id, first_name, last_name).* T