Re: Checkpoint Error

Hi All, Any suggestions? Thanks On Mon, Jan 18, 2021 at 7:38 PM Navneeth Krishnan wrote: > Hi All, > > We are running our streaming job on flink 1.7.2 and we are noticing the > below error. Not sure what's causing it, any pointers would help. We have > 10 TM's checkpointing to AWS EFS. > > Asy

Re: Dynamic JDBC Sink Support

Hey Rion, I had exactly the same problem and implemented this functionality in my Flink fork with XA sink taken from the development branch. As I see that it's not only my problem, I've created a Jira task for it - FLINK-21643 - and will provide draft PR for it. @David - for traditional relationa

Re: reading file from s3

I had a typo in my previous answer, the env name was missing an 'S' ENABLE_BUILT_IN_PLUGIN --> ENABLE_BUILT_IN_PLUGINS once again, the value is the plugin jar name : flink-s3-fs-hadoop-.jar The complete list can be found here

Re: reading file from s3

Hey Avi, Do you use 'Hadoop S3 plugin' to read from S3? If yes, what is its version? If not try to read from S3 as follow (ref) 1. set an environment variable to use hadoop plugin (it's pa

Re: Running Pyflink job on K8s Flink Cluster Deployment?

Hi Kevin, For your information, bellow is an example for running a PyFlink table API WordCount job. 1. Building a Docker image with Python and PyFlink Installed: Dockerfile: FROM flink:1.12.0 # install python3 and pip3 RUN apt-get update -y && \ apt-get install -y python3.7 python3-pip python

Re: Running Pyflink job on K8s Flink Cluster Deployment?

Hi Kevin, You are able to run PyFlink applications on kuberetes cluster, both native k8s mode and resource definition mode are supported since release-1.12.0. Currently, Python and PyFlink are not enabled in official flink docker image, that you might need to build a custom image with Python and P

java options to generate heap dump in EMR not working

Hi, I am trying to generate a heap dump to debug a GC overhead OOM. For that I added the below java options in flink-conf.yaml, however after adding this the yarn is not able to launch the containers. The job logs show it goes on requesting for containers from yarn and it gets them, again releases