[
https://issues.apache.org/jira/browse/FLINK-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451771#comment-16451771
]
ASF GitHub Bot commented on FLINK-8715:
---------------------------------------
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/5885
A note about previous tests that should have been failing because of this
issue:
The tests
`StateBackendTestBase#testKryoRestoreResilienceWithDifferentRegistrationOrder()`
and
`StateBackendTestBase#testPojoRestoreResilienceWithDifferentRegistrationOrder`
was supposed to, to some extend, test serializer reconfigurations across
backend state restores.
f553d9b adds an extension to the Kryo test that verifies the state handles
actually uses the correct reconfigured serializer. After adding that extension,
the test fails without this PR.
However, that still doesn't explain why, especially in the RocksDB case,
modifications to the state did not cause errors while it was using the
non-reconfigured serializer (with outdated Kryo registration mappings).
> RocksDB does not propagate reconfiguration of serializer to the states
> ----------------------------------------------------------------------
>
> Key: FLINK-8715
> URL: https://issues.apache.org/jira/browse/FLINK-8715
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.3.2
> Reporter: Arvid Heise
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.5.0
>
>
> Any changes to the serializer done in #ensureCompability are lost during the
> state creation.
> In particular,
> [https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBValueState.java#L68]
> always uses a fresh copy of the StateDescriptor.
> An easy fix is to pass the reconfigured serializer as an additional parameter
> in
> [https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L1681]
> , which can be retrieved through the side-output of getColumnFamily
> {code:java}
> kvStateInformation.get(stateDesc.getName()).f1.getStateSerializer()
> {code}
> I encountered it in 1.3.2 but the code in the master seems unchanged (hence
> the pointer into master). I encountered it in ValueState, but I suspect the
> same issue can be observed for all kinds of RocksDB states.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
