[
https://issues.apache.org/jira/browse/FLINK-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447724#comment-16447724
]
ASF GitHub Bot commented on FLINK-8715:
---------------------------------------
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/5885#discussion_r183309701
--- Diff:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -1177,7 +1177,7 @@ private void
restoreKeyGroupsShardWithTemporaryHelperInstance(
throw new StateMigrationException("State
migration isn't supported, yet.");
} else {
stateInfo.f1 = newMetaInfo;
- return stateInfo.f0;
+ return Tuple2.of(stateInfo.f0,
newMetaInfo.getStateSerializer());
--- End diff --
I agree here, that we should let the meta info be immutable, and let the
compatibility check result carry the compatible, reconfigured serializer.
However, one issue is that this requires changes to the
`CompatibilityResult` interface which is part of the public API. I would prefer
not to touch the API now as we're approaching release. It would be possible to
by-pass this by maybe introducing an internal compat result class, but
downsides are - 1) that would have almost identical implementation to
`CompatibilityResult`, and 2) that would entail touching a lot of our more
complex serializer's code, because they use
`CompatibilityUtil.resolveCompatibilityResult`.
> RocksDB does not propagate reconfiguration of serializer to the states
> ----------------------------------------------------------------------
>
> Key: FLINK-8715
> URL: https://issues.apache.org/jira/browse/FLINK-8715
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.3.2
> Reporter: Arvid Heise
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.5.0
>
>
> Any changes to the serializer done in #ensureCompability are lost during the
> state creation.
> In particular,
> [https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBValueState.java#L68]
> always uses a fresh copy of the StateDescriptor.
> An easy fix is to pass the reconfigured serializer as an additional parameter
> in
> [https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L1681]
> , which can be retrieved through the side-output of getColumnFamily
> {code:java}
> kvStateInformation.get(stateDesc.getName()).f1.getStateSerializer()
> {code}
> I encountered it in 1.3.2 but the code in the master seems unchanged (hence
> the pointer into master). I encountered it in ValueState, but I suspect the
> same issue can be observed for all kinds of RocksDB states.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
