Hi, using the ValueState and RocksDB to store a map inside the value state means that you will have a different map for each key, which is automatically swapped on a per record basis, depending on the record’s key. If you are using a map and Checkpointed, there is only one map and your code is responsible for dispatching state between different keys.
If you use a map and Checkpointed, the map will be on the heap and the checkpoint will go directly against the filesystem; this is independent of the chosen backend, so no RocksDB is involved. On a further note, we are working on an alternative to ValueState that is like a MapState. In contrast to ValueState, MapState does not deserialize the whole map on each access, but can access individual key/value pairs. This might be what you are looking for. Best, Stefan > Am 01.12.2016 um 09:35 schrieb Anirudh Mallem <anirudh.mal...@247-inc.com>: > > Hi Everyone, > I am trying to understand the Working With State feature page of the Flink > documentation. > My question is in case I am using a ValueState in my CustomMap class to > store my states with the RocksDb as my state backend then it is clear that > every state value is stored in RocksDb. > Now instead of a ValueState if I just use a normal Java Hashmap to store my > states and implement the Checkpointed interface then will the entire HashMap > reside on the RocksDb backend or will the HashMap be in memory and just the > snapshots sent to RocksDb? I am trying to see what will I lose/gain if I have > my own data structure to do state maintenance. Thanks. > > Regards, > Anirudh
