Thank you! This is a great development. On Mon, Nov 25, 2024 at 6:21 AM LDesire <two_som...@icloud.com> wrote:
> Hello, > > I have implemented stateful streaming functionality in the Spark Runner > and am currently in the testing phase. > > Since implementing timer functionality seems quite challenging at this > point, I am considering creating a PR with state management implementation > only. I have added logic to reject pipelines where timers are detected in > the user's DoFn. > > I would like to get the community's opinion on this approach: > 1. Is it acceptable to submit a PR with state-only implementation at this > stage? > Definitely! (in fact when I implemented for Dataflow I did the same thing) > 2. Or should the implementation include both state and timer functionality > to be considered complete? > As long as the runner rejects pipelines that it cannot run, like you say, then it is fine. It is true that there are not too many uses for state without timers, but there are some. And anyhow, doing it in smaller pieces is smart. Kenn > > The current implementation: > - Fully supports state management in streaming context > - Includes validation to reject pipelines that use timers > - Has been tested for state management scenarios > > I believe this could be a good first step, allowing for timer > implementation to be added later. However, I wanted to check with the > community before proceeding with the PR. > > Thank you for your guidance. > > Best regards
