[
https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415014#comment-15415014
]
ASF GitHub Bot commented on FLINK-4340:
---------------------------------------
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2345
@wenlong88 I was talking about this effort to enable key-group sharding in
Flink: https://issues.apache.org/jira/browse/FLINK-3755
With this is becomes necessary to checkpoint the keyed states in a way that
operators can restore the state for the key shard for which they are
responsible. The state backend therefore has to write the snapshot data such
that keys for different key-groups are separated. If we keep the approach of
simply checkpointing the whole RocksDB DB we don't get this property. In the
fully-async checkpointing mode we iterate over the keys for each key group and
write them separately from the keys of the other key groups. (logically
speaking. they can still go into the same file but we need an index that tells
us where the state for each key group resides.)
> Remove RocksDB Semi-Async Checkpoint Mode
> -----------------------------------------
>
> Key: FLINK-4340
> URL: https://issues.apache.org/jira/browse/FLINK-4340
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Affects Versions: 1.1.0
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
>
> This seems to be causing to many problems and is also incompatible with the
> upcoming key-group/sharding changes that will allow rescaling of keyed state.
> Once this is done we can also close FLINK-4228.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
