Hi Gyula, Thanks for your quick response!
1. Instead of having to implement a method in the interface could we > instead use an empty marker interface such as "WithAsyncStateAccess" > Well it's a good suggestion, I changed the FLIP accordingly. > 2. If a function is marked async can the user use the sync methods of > the new v2 state api (v2State.value()), will this still work? What is > the > performance implication? > Yes, it still works. The V2 API provides both sync and async methods that can be mixed. The performance of sync methods from state V2 are not as good as the state V1, since the operator/executor should handle some potential access order issues raised by mixed usage of async & sync API, which takes some CPU overhead. In future we may introduce a global force option like 'state.force-sync: true'. If set, all the async state access will run in sync mode, and we can safely skip the async control logic and provide the best performance for sync methods. > 3. Why do we need both a sync and async version of the reducer/min/max > logic? Could they always use the new async interface? What is the > performance implication? I think we still need the sync version with state V1 APIs for the current version. The asynchronous state access cannot take advantage when the state is small and mostly in memory. The overhead of asynchronous flow control is considerable compared to the state access itself. In the future we can consider supporting automatic switching from sync implementation to async version when the state reaches a certain size, for those reducer/min/max methods. Best, Zakelly On Wed, Dec 4, 2024 at 9:36 PM Gyula Fóra <gyula.f...@gmail.com> wrote: > Hey! > > Some questions / comments: > > > 1. Instead of having to implement a method in the interface could we > instead use an empty marker interface such as "WithAsyncStateAccess" > 2. If a function is marked async can the user use the sync methods of > the new v2 state api (v2State.value()), will this still work? What is > the > performance implication? > 3. Why do we need both a sync and async version of the reducer/min/max > logic? Could they always use the new async interface? What is the > performance implication? > > Thanks! > Gyula > > On Wed, Dec 4, 2024 at 1:33 PM Zakelly Lan <zakelly....@gmail.com> wrote: > > > Hi everyone, > > > > I'd like to start a discussion on FLIP-488: Expose Async State Processing > > and New State APIs in Datastream(V1) APIs[1]. > > > > This FLIP allows Datastream users to leverage the capability of async > state > > query and disaggregated state management introduced in FLIP-423[2] and > its > > sub-FLIPs. > > > > Looking forward to hearing from you! > > > > [1] https://cwiki.apache.org/confluence/x/yIrREw > > [2] https://cwiki.apache.org/confluence/x/R4p3EQ > > > > > > Best, > > Zakelly > > >
