Hi Folks,
We are currently running a Java 8 application that uses Protobufs and
runs on Apache Flink 1.18.1. While Flink supports the Protobuf data format
for message exchange between operators, the serialization and
deserialization of these Protobuf messages are performed using Kryo 2.x for
JDK 8. To upgrade our application for Java 17 compatibility, supported as
of Flink 1.18.1, the Kryo library in Flink must be updated from version
2.24.0 to 5.5.0, as Kryo 2.24.0 does not support Java 17. This improvement
is being tracked under FLIP-317: Upgrade Kryo from 2.24.0 to 5.5.0. We
would like to confirm if the upcoming major release, Apache Flink 2.0, will
include support for Kryo 5 as part of this improvement plan.
The following references outline the changes necessary for introducing
Kryo 5 support: https://github.com/apache/flink/pull/22660
https://issues.apache.org/jira/browse/FLINK-3154
https://github.com/twitter/chill/issues/692
https://github.com/EsotericSoftware/kryo/issues/885
Additional Context: Currently, all Flink applications using Protobuf
generate state with Kryo v2. Once the proposed improvement is implemented,
Flink applications will fully support reading existing state with Kryo v2
and writing new state with Kryo v5. However, the latest Kryo v5 does not
support snapshots created by the older Kryo v2. This limitation means that
applications relying on snapshots will need to use an intermediate "bridge
version" of Flink running on Java 11. The bridge version will allow these
applications to read state created by Kryo v2 and rewrite it using Kryo v5
before upgrading to a future Flink version that drops support for Kryo v2
entirely.
We are seeking clarification on whether Flink 2.0 will include these
updates, ensuring a smooth upgrade path for applications moving to Kryo 5.
Thank you for your time and support!
