Hi there, I’ve got a MapState where I need to iterate over the entries.
This currently isn’t supported (at least for Rocks DB), AFAIK, though there is an issue/PR <https://issues.apache.org/jira/browse/FLINK-8297> to improve this. The best solution I’ve seen is what Fabian proposed, which involves keeping a ValueState with a count of entries, and then having the key for the MapState be the index. > I cannot comment on the internal design, but you could put the data into a > RocksDBStateBackend MapState<Integer, X> where the value X is your data > type and the key is the list index. You would need another ValueState for > the current number of elements that you put into the MapState. > A MapState allows to fetch and traverse the key, value, or entry set of the > Map without loading it completely into memory. > The sets are traversed in sort order of the key, so should be in insertion > order (given that you properly increment the list index). This effectively lets you iterate over all of the map entries for a given (keyed) state - though it doesn’t solve the “I have to iterate over _every_ entry” situation. Is this currently the best option? Thanks, — Ken -------------------------------------------- http://about.me/kkrugler +1 530-210-6378
