[no subject]

Unsubscribe

Broadcasting feature not working

Hi Team, We are experiencing a very weird issue recently which relates to the flink state broadcasting feature. The issue is sometimes we see the Flink Job is not pulling the control stream upon job submission. The below code snippet which is responsible to pull the configs stored in a S3 bucket

Enable Multiple kafka Consumer sources for job

Hi Team , We are trying to build a data pipeline where we have to set up two different kafka consumers for two different kafka topics and with a single SNS sink. Below is the sample code for the same , but looks like from one of the sources the events are not flowing into the cluster. We are using

Need Clarity about Checkpoint for Flink-1.12.2

Hi Team, We have recently enabled Check Pointing in our flink job using FSStateBackend pointing to S3 bucket. Below is the sample code for enabling check pointing though app code and we are using flink version - 1.12.2 . env.setStateBackend(new FsStateBackend("s3://flinkcheckpointing/job-name/",

Re: Unused Checkpointed folder in S3

t; if you don't set RETAIN_ON_CANCELLATION, the folder should be cleaned up > automatically. If you explicitly want to retain the checkpoint, then there > is not much that Flink can do or I may have misunderstood you. > > On Tue, May 11, 2021 at 4:09 PM sudhansu jena > wrote: > >> Hi Team,

Re: Unused Checkpointed folder in S3

The flink version we are using is 1.12.2 Thanks, Sudhansu On Tue, May 11, 2021 at 7:48 PM Chesnay Schepler wrote: > Which Flink version are you using? > > On 5/11/2021 4:09 PM, sudhansu jena wrote: > > Hi Team, > > > > We have recently enabled Check Poin

Unused Checkpointed folder in S3

Hi Team, We have recently enabled Check Pointing in our flink job using FSStateBackend pointing to S3 bucket. Below is the sample code for enabling the checkpointing for the job. The query is each time we cancel the job and restart from the flink dashboard, a new folder is getting created along w

Re: Job failing after enabling Checkpointing

apache/flink/api/common/ExecutionConfig.html#addDefaultKryoSerializer-java.lang.Class-java.lang.Class > - > > > https://github.com/apache/flink/blob/release-1.12.2/flink-core/src/main/java/org/apache/flink/api/common/ExecutionConfig.java#L730 > > On 5/10/2021 11:11 AM, sudhansu jena

Re: Job failing after enabling Checkpointing

https://stackoverflow.com/questions/32453030/using-an-collectionsunmodifiablecollection-with-apache-flink > > On 5/10/2021 10:48 AM, sudhansu jena wrote: > > Hi Team, > > We have recently enabled check pointing in our flink job using S3 as the > state backend, but while submit

Job failing after enabling Checkpointing

Hi Team, We have recently enabled check pointing in our flink job using S3 as the state backend, but while submitting the Job, it fails with the below error.Can you please let us know what is going wrong here. Below is the code snippet for enabling check pointing. env.setStateBackend(new

Enabling Checkpointing using FsStatebackend

Hi Team, We have recently enabled checking pointing using FsStateBackend where we are trying to use S3 bucket as the persistent storage but after enabling it we are running into issues while submitting the job into the cluster. Can you please let us know if we are missing anything ? Below is th