Re: ContinuousFileMonitoringFunction retrieved invalid state.

Hi, Are you using the reactive mode? There is a known issue like that: https://issues.apache.org/jira/browse/FLINK-28274 Best, Lijie yuxia 于2022年7月1日周五 09:49写道： > I'm not sure why it happened. But from the Flink source code, it seems try > to restore from an invalid state. Seems the state actu

Re: ContinuousFileMonitoringFunction retrieved invalid state.

I'm not sure why it happened. But from the Flink source code, it seems try to restore from an invalid state. Seems the state actually contains more that one value, but Flink expected the state should contains one or zero value. Best regards, Yuxia 发件人: "Vishal Surana" 收件人: "User" 发送时间:

Re: StreamingFileSink & checkpoint tuning

Streaming file sink will write to s3 when processing element. But it's just temporary file. Only after one successful checkpoint (more exactly, once recieve a notification for successful checkpoint), will it commit these temporary files written since last successful checkpoint . Best regards,

ContinuousFileMonitoringFunction retrieved invalid state.

My job is unable to restore state after savepoint due to the following exception. Seems to be a rare exception as I haven't found any forum discussing it. Please advise. java.lang.IllegalArgumentException: ContinuousFileMonitoringFunction retrieved invalid state. at org.apache.flink.util.Precondit

StreamingFileSink & checkpoint tuning

Hi, I recently encountered an issue while using StreamingFileSink. I have a flink job consuming records from various sources and write to s3 with streaming file sink. But the job sometimes fails due to checkpoint timeout, and the root cause is checkpoint alignment failure as there is data skewnes

How to store AvroSchema POJOs in state-store ?

Hi , I am unable to store Avro-schema based events in the state store, I am using the following configurations. I see the doc says, currently Avro is supported by defau

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

Hi Mason, I added mason6345 to the Flink confluence space, you should be able to add a FLIP now. Looking forward to the contribution! Thomas On Thu, Jun 30, 2022 at 9:25 AM Martijn Visser wrote: > > Hi Mason, > > I'm sure there's a PMC (*hint*) out there who can grant you access to > create a

Re: influxdb metrics reporter - 4k series per job restart

Hi, Filip You can modify the InfluxdbReporter code to rewrite the notifyOfAddedMetric method and filter the required metrics for reporting. Best, Weihua On Thu, Jun 30, 2022 at 8:46 PM Filip Karnicki wrote: > Hi All > > We're using the influx reporter (flink 1.14.3), which seems to create a >

Re: RuntimeContextInitializationContextAdapters: ClassNotFoundException

Hi, Harald Which version of flink are you referring to? Best, Weihua On Thu, Jun 30, 2022 at 11:38 AM Harald Busch wrote: > I get an java.lang.ClassNotFoundException: > org.apache.flink.api.common.serialization.RuntimeContextInitializationContextAdapters > when running my Apache Flink code lo

Flink Elasticsearch connector 7 with SSL

Hi Flink community members, In my flink job, I am trying to use *elasticsearch7* connector. I could not find a way to communicate over SSL when using *Elasticsearch7SinkBuilder* Am I missing something? Also there is an deprecated class *ElasticsearchSink.Builder* where it has *RestClientFactory*

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

Hi Mason, I'm sure there's a PMC (*hint*) out there who can grant you access to create a FLIP. Looking forward to it, this sounds like an improvement that users are looking forward to. Best regards, Martijn Op di 28 jun. 2022 om 09:21 schreef Mason Chen : > Hi all, > > Thanks for the feedback!

influxdb metrics reporter - 4k series per job restart

Hi All We're using the influx reporter (flink 1.14.3), which seems to create a series per: -[task|job]manager - host - job_id - job_name - subtask_index - task_attempt_id - task_attempt_num - task_id - tm_id which amounts to about 4k of series each time our job restarts itself We are currently e