What you want is a Flink 1.x to Flink 2.x migration tool; that's much more
complicated than simply migrating Kryo because there were lots of compatibility
breaking changes beyond the Kryo upgrade.
Flink 1.x used Kryo by default everywhere. Flink 2.0 does not. That's a
compatibility breaking cha
Kurt Ostfeld created FLINK-37404:
Summary: Reenable Migration Tests for Filnk 2.x
Key: FLINK-37404
URL: https://issues.apache.org/jira/browse/FLINK-37404
Project: Flink
Issue Type: Technical
A state migration tool that can migrate savepoints/checkpoints is a great idea
that would be very useful for a lot of scenarios. But that is a bigger scope
issue.
(fyi, CI is passing and PR should be fully ready)
The only reason to keep the old version of Kryo and not either upgrade or
remove Kryo is if there is a backward compatibility advantage. If backward
compatibility is breaking anyway, this seems an easy choice.
Java record support was the big motivation to this upgrade. From some simple
tests I'
Kurt Ostfeld created FLINK-33866:
Summary: KafkaSinkBuilder in flink-connector-kafka references
DeliveryGuarantee in flink-connector-base
Key: FLINK-33866
URL: https://issues.apache.org/jira/browse/FLINK-33866
apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>
> On Sun, Jun 18, 2023 at 1:18 AM Kurt Ostfeld kurtostf...@proton.me.invalid
>
> wrote:
>
> > I built the Flink master branch and tried running this simple Flink app
> > that uses a Java record:
> >
I built the Flink master branch and tried running this simple Flink app that
uses a Java record:
https://github.com/kurtostfeld/flink-kryo-upgrade-demo/blob/main/flink-record-demo/src/main/java/demo/app/Main.java
It fails with the normal exception that Kryo 2.x throws when you try to
serialize
Regarding this comment: "The version in the state is the serializer version,
and applies to the entire state, independent of what it contains. If you use
Kryo2 for reading and Kryo5 for writing (which also implies writing the new
serializer version into state), then I'd assume that a migration i
> On 08/06/2023 16:06, Kurt Ostfeld wrote:
>
> > If I understand correctly, the scenario is resuming from multiple
> > checkpoint files or from a savepoint and checkpoint files which may be
> > generated by different versions of Flink
>
>
> No; it's the
> > > PR yet...)
> > >
> > > What we kind of want is this:
> > > 1) Kryo5 is used as the default for new jobs. (maybe not even that,
> > > making it an explicit opt-in)
> > > 2) Kryo2 is used for reading AND writing for existing* jobs by default.
opt-in)
> 2) Kryo2 is used for reading AND writing for existing* jobs by default.
> 3) Users can explicitly (and easily!) do a full migration of their jobs,
> after which 2) should no longer apply.
>
>
>
> In the PR you mentioned running into issues on Java 17; to have have
> som
ryo to a newer version - if you’ve solved that, kudos.
>
> — Ken
>
>
> > On May 29, 2023, at 2:21 PM, Kurt Ostfeld kurtostf...@proton.me.INVALID
> > wrote:
> >
> > Hi everyone. I would like to start the discussion thread for FLIP-317:
> > Upgrade Kryo fro
Hi everyone. I would like to start the discussion thread for FLIP-317: Upgrade
Kryo from 2.24.0 to 5.5.0 [1].
There is a pull-request associated with this linked in the FLIP.
I'd particularly like to hear about:
- Chesnay Schepler's request to consider removing Kryo serializers from the
execut
Chesnay Schepler asked me to create a FLIP for this pull request:
https://github.com/apache/flink/pull/22660
I created an account for the Flink Confluence site with username "kurto", but I
don't have access to create pages, and therefore don't have access to create a
FLIP. I see the FLIP docs sa
Kurt Ostfeld created FLINK-32104:
Summary: stop-with-savepoint fails and times out with simple
reproducible example
Key: FLINK-32104
URL: https://issues.apache.org/jira/browse/FLINK-32104
Project
Kurt Ostfeld created FLINK-31938:
Summary: Failing Unit Test: FlinkConnectionTest.testCatalogSchema
"Failed to get response for the operation"
Key: FLINK-31938
URL: https://issues.apache.org/jira/br
Kurt Ostfeld created FLINK-31937:
Summary: Failing Unit Test: ClientTest.testClientServerIntegration
"Connection leak"
Key: FLINK-31937
URL: https://issues.apache.org/jira/browse/FLINK-31937
Kurt Ostfeld created FLINK-31897:
Summary: Failing Unit Test:
org.apache.flink.queryablestate.network.ClientTest.testRequestUnavailableHost
Key: FLINK-31897
URL: https://issues.apache.org/jira/browse/FLINK-31897
Kurt Ostfeld created FLINK-31880:
Summary: Bad Test in OrcColumnarRowSplitReaderTest
Key: FLINK-31880
URL: https://issues.apache.org/jira/browse/FLINK-31880
Project: Flink
Issue Type: Bug
19 matches
Mail list logo
