Hi everyone, I'd like to start a discussion of FLIP-538: Support Custom Generic Type Serializer [1].
Flink handles most common types efficiently with the built-in serializers and leaves the remaining to the generic type serializer Kryo in a hardcoded way. We propose supporting custom generic type serializers with two major motivations: 1. State-compatibility: Flink 2.0 upgraded Kryo from v2.24.0 to v5.6.2 with FLIP-317 [2], and hence dropped state compatibility with 1.x jobs that rely on Kryo v2.24.0. Users can copy the impl from previous Flink release for a smooth upgrade. 2. Performance: new serializer frameworks like Apache Fory [3] provides 3x speedup compared to Kryo by our internal testing. We also plan to cooperate with the Fory community to maintain an implementation under the Fory repo. Looking forward to suggestions on it! [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP+538%3A+Support+Custom+Generic+Type+Serializer [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 [3] https://fory.apache.org/ Best, Zhanghao Chen
