Thanks for the reply. Yes, it's true that RocksDB only supports bytes-comparator which makes it challenging. I checked FLINK-6219 has already been closed with reason "The thing is this issue will be addressed after blink code gets merged, and we will open another Jira to track the requirements. " However, it looks like it's still not resolved yet. Not sure if the blink merge is still ongoing. Could we bring this back to the table?
Best, Sean On Sat, Jul 25, 2020 at 4:56 AM Congxian Qiu <qcx978132...@gmail.com> wrote: > Hi Sean > AFAIK, Flink does not support SortedMapState currently, there was an > issue FLINK-6219 about sorted mapstate before. I think you can reach to > this issue if you want sorted mapstate. > Sorted MapState for RocksDB can be challenging for Java-based comparator, > RocksDB supports the bytes-comparator only. > > Best, > Congxian > > > Sean Z <xingha...@gmail.com> 于2020年7月24日周五 上午3:36写道： > > > Hi Flink Community, > > > > I'm new to this community but have been using Flink for a year or so. As > a > > user, my team built stuff upon Flink stateful stream processing. We used > to > > have a use case that we need a sorted map data structure to store our > > state, something like TreeMap in Java, to query higher/lower keys, do > range > > queries, and etc. However, currently, we only have a MapState interface > in > > Flink. We found RocksDB is kind of sorted storage in nature. To achieve > > our use case, we have to do some hacky tricks to bypass current > limitations > > and use lower RocksDB features as a workaround to implement our own > > SortedMapState. I assume that there should be lots of other users who > have > > the same use case, so I have a few questions here. > > > > 1. Do we have a feature like SortedMapState already in place, in > > development or in the future roadmap? > > > > 2. If not, could that be a good feature to have? and are there any other > > concerns? > > > > Just want to start a discussion here from a user perspective. If > everything > > goes well, we are also interested in contributing back to the community. > > > > > > Best regards, > > Sean > > >
