Hi Gyula,
thanks for bringing this up. I agree that we should use one of the most
recent Kryo versions for Flink 2.0. Otherwise the community will suffer
again from incompatibilities and needs to wait for another major Flink
version. Thanks for starting this discussion. We should do it for 2.0.
If not now, when?
Cheers,
Timo
On 24.02.25 08:03, Gyula Fóra wrote:
Hey!
I think the main question here is whether Flink 2.0 is going to be state
backward compatible or not.
If not then we need to make the upgrade right now, freeze or not. We have
to decide this as a community.
If we need to preserve backward compatibility then we need to go with a
much more complex solution here when upgrading Kryo to do that.
Do we have a community decision somewhere related to state
backward compatibility for 2.0?
Cheers
Gyula
On Mon, Feb 24, 2025 at 3:02 AM Rui Fan <1996fan...@gmail.com> wrote:
Thanks Gyula for driving this discussion!
+1 for upgrading kyro, I have a question about the timeline.
The flink 2.0.0 has been freezed, do we have enough time to
test if it's done in flink 2.0.0?
Best,
Rui
On Fri, Feb 21, 2025 at 9:55 PM Alexander Fedulov <afedu...@apache.org>
wrote:
Hi Gyula,
Thanks for bringing up this topic! Kryo incompatibility with newer
Java versions is a major issue that needs to be addressed, and in my
opinion, Flink 2.0 provides a great opportunity to introduce this
change.
My understanding is that state compatibility was not a strict goal
during 2.0 development. However, I’ve heard mentions that it might
actually be compatible after all.
From this perspective:
definitely +1 on upgrading Kryo. The decision to invest additional
effort in maintaining compatibility should depend on whether all other
changes have preserved compatibility guarantees. If Kryo is the only
breaking change, then ensuring compatibility might be worth
considering.
Best,
Alex
On Fri, 21 Feb 2025 at 06:05, Gyula Fóra <gyula.f...@gmail.com> wrote:
Hey all!
I would like to rekindle this discussion as it seems that it has
stalled
several times in the past and we are nearing the point in time where
the
decision has to be made with regards to 2.0. (we are already a bit late
but
nevermind)
There has been numerous requests and efforts to upgrade Kryo to better
support newer Java versions and Java native types. I think we can all
agree
that this change is inevitable one way or another.
The latest JIRA for this seems to be:
https://issues.apache.org/jira/browse/FLINK-3154
There is even an open PR that accomplishes this (currently in a state
incompatible way) but based on the discussion it seems that with some
extra
complexity compatibility can even be preserved by having both the old
and
new Kryo versions active at the same time.
The main question here is whether state compatibility is important for
2.0
with this regard or we want to bite the bullet and make this upgrade
once
and for all.
Cheers,
Gyula
