Hi Kurt, I personally think it’s a very nice improvement, and that the longer-term goal of removing built-in Kryo support for state serialization (while a good one) warrants a separate FLIP.
Perhaps an intermediate approach would be to disable the use of Kryo for state serialization by default, and force a user to disregard warnings and explicitly enable it if they want to go down that path. I’d assumed that there wasn’t a good way to migrate state stored with an older version of Kryo to a newer version - if you’ve solved that, kudos. — Ken > On May 29, 2023, at 2:21 PM, Kurt Ostfeld <kurtostf...@proton.me.INVALID> > wrote: > > Hi everyone. I would like to start the discussion thread for FLIP-317: > Upgrade Kryo from 2.24.0 to 5.5.0 [1]. > > There is a pull-request associated with this linked in the FLIP. > > I'd particularly like to hear about: > > - Chesnay Schepler's request to consider removing Kryo serializers from the > execution config. Is this a reasonable task to add into this FLIP? Is there > consensus on how to resolve that? Would that be better addressed in a > separate future FLIP after the Kryo upgrade FLIP is completed? > > - Backwards compatibility. The automated CI tests have a lot of backwards > compatibility tests that are passing. I also wrote a Flink application with > keyed state using custom Kryo v2 serializers and then an upgraded version > with both Kryo v2 and Kryo v5 serializers to stress test the upgrade process. > I'd like to hear about additional scenarios that need to be tested. > > - Is this worth pursuing or is the Flink project looking to go in a different > direction? I'd like to do some more work on the pull request if this is being > seriously considered for adoption. > > I'm looking forward to hearing everyone's feedback and suggestions. > > Thank you, > Kurt > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 -------------------------- Ken Krugler http://www.scaleunlimited.com Custom big data solutions Flink, Pinot, Solr, Elasticsearch
