Re: java.lang.UnsupportedOperationException: A serializer has already been registered for the state; re-registration is not allowed.

2024-01-17 Thread Zakelly Lan
Hi, Could you please share the code of state initialization (getting state from a state descriptor)? It seems you are creating a state in #processElement? Best, Zakelly On Thu, Jan 18, 2024 at 2:25 PM Zakelly Lan wrote: > Hi, > > Could you please share the code of state initialization (gettin

RE: Flink Slow Execution

2024-01-17 Thread Jiabao Sun
Hi Dulce, MiniCluster is generally used for local testing and is limited by the resources of a single machine. When more tasks are executed, it may not be able to immediately acquire the resources needed to start the MiniCluster, resulting in slower startup times. If running Flink tasks in a

Flink Slow Execution

2024-01-17 Thread Dulce Morim
Hello, In a single JVM, I'm running multiple flink batch jobs locally using the MiniCluster (Java 17 and Flink 1.18). At the beginning of the process, the Mini Cluster starts pretty much instantly. However, I'm running into an issue where the more jobs I execute the longer the MiniCluster take

Watermark advancing too quickly when reprocessing events with event time from Kafka

2024-01-17 Thread adrianbartnik.mailbox.org via user
Hi everyone, we are struggling to understand how Flink handles watermarks from Kafka when reprocessing events using their event time. The goal of the Flink job is to consume events from 3 Kafka topics (3 partitions each), order them using their event time (using a process function) and write t

java.lang.UnsupportedOperationException: A serializer has already been registered for the state; re-registration is not allowed.

2024-01-17 Thread Konstantinos Karavitis
Have you ever met the following error when a flink application restarts and tries to restore the state from RocksDB? *Caused by: java.lang.UnsupportedOperationException: A serializer has already been registered for the state; re-registration is not allowed. at org.apache.flink.runtime.state.State