Tzu-Li (Gordon) Tai created FLINK-11925:
-------------------------------------------
Summary: KryoSerializerSnapshot doesn't completely capture state /
configuration of Kryo instance
Key: FLINK-11925
URL: https://issues.apache.org/jira/browse/FLINK-11925
Project: Flink
Issue Type: Bug
Components: API / Type Serialization System
Affects Versions: 1.7.2, 1.6.4, 1.8.0
Reporter: Tzu-Li (Gordon) Tai
Currently, the {{KryoSerializerSnapshot}} only covers information about
registered types / serializers that was configured in the {{ExecutionConfig}}.
This is problematic, because there are a few cases where we have some
additional registrations:
1) When Avro is present in the classpath [1] [2]
2) When Scala is used, in which case Twitter Chill is used which itself has
some registrations [3]
3) If a non-registered type is encountered, Kryo will on-the-fly registered the
type because we currently configure Kryo to allow dynamic registrations [4].
For case 1), we do reflect these additional registrations in the
{{KryoSerializerSnapshot}}.
This isn't the case for 2) and 3), which would cause problems when attempting
to create a reconfigured instance of the {{KryoSerializer}} on restore.
In general, instead of relying on trying to keep track of the registrations
ourselves, it would be much more straightforward if there is a way to "dump"
the state / configuration of Kryo when we attempt to create a snapshot of the
{{KryoSerializer}}.
Whether or not Kryo has APIs to allow this needs further investigation.
[1]
https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/utils/AvroKryoSerializerUtils.java#L51
[2]
https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/utils/AvroKryoSerializerUtils.java#L68
[3]
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java#L430
[4]
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java#L476
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
