[
https://issues.apache.org/jira/browse/FLINK-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai updated FLINK-11771:
----------------------------------------
Description:
This is a upgrade path that was overlooked in
{{TypeSerializerSnapshot#readVersionedSnapshot}}.
If the serializer snapshot was a {{TypeSerializerConfigSnapshot}} when it was
written in 1.7+ Flink versions, the actual snapshot content will be prefixed by
a magic {{TypeSerializerConfigSnapshot#ADAPTER_VERSION}} int as the snapshot
version, as well as Java-serialized prior serializer.
If when restoring, the serializer snapshot was upgraded in-place to a
{{TypeSerializerSnapshot}} (in-place meaning, same classname, not introducing a
new snapshot class),
{{TypeSerializerSnapshot#readVersionedSnapshot}} doesn't work as expected.
Firstly, the provided {{readVersion}} to the user-implemented
{{TypeSerializerSnapshot#readSnapshot}} method would be the magic
{{ADAPTER_VERSION}}.
Secondly, the remaining stream would contain the Java-serialized prior
serializer, which is no longer for the user (because they already upgraded to
the new abstraction).
was:
This is a upgrade path that was overlooked in
{{TypeSerializerSnapshot#readVersionedSnapshot}}.
If the serializer snapshot was a {{TypeSerializerConfigSnapshot}} when it was
written in 1.7+ Flink versions, the actual snapshot content will be prefixed by
a magic {{TypeSerializerConfigSnapshot#ADAPTER_VERSION}} int as the snapshot
version, as well as Java-serialized prior serializer.
If when restoring, the serializer snapshot was upgraded in-place to a
{{TypeSerializerSnapshot}},
{{TypeSerializerSnapshot#readVersionedSnapshot}} doesn't work as expected.
Firstly, the provided {{readVersion}} to the user-implemented
{{TypeSerializerSnapshot#readSnapshot}} method would be the magic
{{ADAPTER_VERSION}}.
Secondly, the remaining stream would contain the Java-serialized prior
serializer, which is no longer for the user (because they already upgraded to
the new abstraction).
> Serializer snapshot cannot be read if directly upgraded in-place to a
> TypeSerializerSnapshot from a TypeSerializerConfigSnapshot written in 1.7+
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-11771
> URL: https://issues.apache.org/jira/browse/FLINK-11771
> Project: Flink
> Issue Type: Improvement
> Components: API / Type Serialization System
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.8.0
>
>
> This is a upgrade path that was overlooked in
> {{TypeSerializerSnapshot#readVersionedSnapshot}}.
> If the serializer snapshot was a {{TypeSerializerConfigSnapshot}} when it was
> written in 1.7+ Flink versions, the actual snapshot content will be prefixed
> by a magic {{TypeSerializerConfigSnapshot#ADAPTER_VERSION}} int as the
> snapshot version, as well as Java-serialized prior serializer.
> If when restoring, the serializer snapshot was upgraded in-place to a
> {{TypeSerializerSnapshot}} (in-place meaning, same classname, not introducing
> a new snapshot class),
> {{TypeSerializerSnapshot#readVersionedSnapshot}} doesn't work as expected.
> Firstly, the provided {{readVersion}} to the user-implemented
> {{TypeSerializerSnapshot#readSnapshot}} method would be the magic
> {{ADAPTER_VERSION}}.
> Secondly, the remaining stream would contain the Java-serialized prior
> serializer, which is no longer for the user (because they already upgraded to
> the new abstraction).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
