Hello Flink Folks, I have been spending some time looking and reading about Flink Stateful Functions and evaluating its usage for different types of streaming processing jobs. I know it is meant for an Event Driven System but I am evaluating it for other use cases like simple stateful stream processing with drop in functions. The idea is to allow product teams to bring their business logic and be able to write data pipelines in less time and require less knowledge compared to using and deploying something like the DataStream API (I am aware of the other APIs like Table, Flink SQL and DataStream interop).
Below are my questions. I want to make sure I understand StateFuns and the core functionality - and potentially some of the future roadmap. 1. KeyBy: From the looks of it, ingress / egress can be scaled, but how does the scaling of the functions work? If those functions have state, do they all have access to all state or is there any keying available for a distributed system (similar to the DataStream API). I have read a large portion of the code and I don't see anything that could resemble this feature but maybe it is being added or I missed it? 2. Source vs SourceFunction: Is there any plan in the roadmap of Stateful Functions to support the source API? I have been writing custom IO modules for internal sources / sinks and think it would be awesome if (for the most part) we could drag and drop Sources as embedded modules with simple wrappers. 3. Does the community know of any companies using StateFuns at scale? Thanks! If you need any clarifications I would be happy to oblige. Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilities [image: Shopify] <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
