Cancelling a job with RMQSource

2022-09-28 Thread Nadia Mostafa
Hello, I have a flink streaming job that consumes data from RMQ source, process it and output it to RMQ sink. I want to delete the RMQ source queue when cancelling the job but keep it if the job failed to resume processing it's data when the job is restarted. One solution is to override the canc

RE: Loading broadcast state on BroadcastProcessFunction instantiation or open method

2022-09-28 Thread alfredo.vasquez.spglobal.com via user
Thank you, I have tried both approaches, Overriding open method did not work but by implementing CheckpointedFunction and overriding initializeState I was able to access and operate over broadcast state @Override public void initializeState(FunctionInitializationContext context) throws Exceptio

Re: Is it practicle to enrich a Flink DataStream in middle operator with Flink Stateful Functions?

2022-09-28 Thread Martijn Visser
Hi Marco, The email is received by the list, but no answers have yet been provided unfortunately. Best regards, Martijn On Wed, Sep 28, 2022 at 4:11 PM Marco Villalobos wrote: > Did this list receive my email? > > I’m only asking because my last few questions have gone unanswered and > maybe

Re: Is it practicle to enrich a Flink DataStream in middle operator with Flink Stateful Functions?

2022-09-28 Thread Marco Villalobos
Did this list receive my email? I’m only asking because my last few questions have gone unanswered and maybe the list server is blocking me. Anybody, please let me know. > On Sep 26, 2022, at 8:41 PM, Marco Villalobos > wrote: > > I indeed see the value of Flink Stateful Functions. > > How

Re: [ANNOUNCE] Apache Flink 1.14.6 released

2022-09-28 Thread Jingsong Li
Thanks Xingbo for releasing it. Best, Jingsong On Wed, Sep 28, 2022 at 10:52 AM Xingbo Huang wrote: > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.14.6, which is the fifth bugfix release for the Apache Flink 1.14 > series. > > Apache Flink® is an open-

Re: Why is task manager shutting down?

2022-09-28 Thread Congxian Qiu
Hi John Yes, the whole TaskManager exited because the task did not react to cancelling signal in time ``` 2022-08-30 09:14:22,138 ERROR org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Task did not exit gracefully within 180 + seconds. org.apache.flink.util.FlinkRuntimeException

Question regarding to debezium format

2022-09-28 Thread Edwin
Hi guys, I was trying to use flink sql to consume data from kafka source, the format of which is debezium-avro-confluent. And I encountered a AvroTypeException saying that "Found something, expecting union", where something is not a type but a field that I defined in the schema registery. So