Inquiry: Kryo 5 Support in Apache Flink 2.0

2024-11-27 Thread Dhruv Patel
Hi Folks, We are currently running a Java 8 application that uses Protobufs and runs on Apache Flink 1.18.1. While Flink supports the Protobuf data format for message exchange between operators, the serialization and deserialization of these Protobuf messages are performed using Kryo 2.x for

Re: Checkpoint failures due to other subtasks sharing the ChannelState file (Caused the Job to Stall)

2024-08-02 Thread Dhruv Patel
We have also enabled unaligned checkpoints. Could it be because of that? We were experience slowness and intermittent packet loss when this issue occurred. On Wed, Jul 31, 2024 at 7:43 PM Dhruv Patel wrote: > Hi Everyone, > > We are observing an interesting issue with continuous c

Checkpoint failures due to other subtasks sharing the ChannelState file (Caused the Job to Stall)

2024-07-31 Thread Dhruv Patel
Hi Everyone, We are observing an interesting issue with continuous checkpoint failures in our job causing the event to not be forwarded through the pipeline. We saw a spam of the below log in all our task manager instances. Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Th