Hi Shimin, Thanks for bring this discussion up.
First of all, I'd like to confirm/clarify that this discussion is mainly about managed state with customized state descriptor rather than raw state, right? Asking because raw state was the very first thing came to my mind when seeing the title. And this is actually the first topic/question we need to discuss, that whether we should support user-defined state descriptor and still ask framework to manage the state life cycle. Personally I'm +1 on this since the "official" state (data-structure) types (currently mainly value, list and map) may not be optimized for customer case, but we'd better ask others' opinion. Secondly, if the result of the first question is "Yes", then it's truly a problem that "Although we can implement customized StateDescriptors for different kind of data structures, we do not really have access to such customized state in RichFunctions", and how to resolve it is the second topic/question to discuss. I've noticed your proposal of exposing "getParitionedState" method out in "RuntimeContext" and "KeyedStateStore" in JIRA (FLINK-14003), but IMO adding a specific interface like below is better than exposing the internal one: <S extends State, V> State getCustomizedState(StateDescriptor<S, V> stateProperties); Finally, I think this is a user-facing and definitely worthwhile discussion, and requires a FLIP to document the conclusion and design/implementation (if any) down. What's your opinion? Thanks. Best Regards, Yu On Fri, 6 Sep 2019 at 13:27, shimin yang <ysmcl...@gmail.com> wrote: > Hi every, > > I would like to start a discussion on supporting customize state > in customized KeyedStateBackend. > > In Flink, users can customize KeyedStateBackend to support different type > of data store. Although we can implement customized StateDescriptors for > different kind of data structrues, we do not really have access to such > customized state in RichFunctions. > > I propose to add a getOtherState method in RuntimeContext and > DefaultKeyedStateStore which directly takes StateDescriptor as parameter to > allow user to get customized state. > > What do you think? > > Thanks. > > Best, > Shimin >
