Re: StatefulDoFnRunner

2017-04-06 Thread Kenneth Knowles
I have some knowledge and opinions about StepContext and *Internals. - StepContext is an interface that basically just grew as "whatever we need right now, add it". It is also mostly left over from before Beam. It is not really the result of conscious holistic design. So no one should feel attach

Re: StatefulDoFnRunner

2017-04-06 Thread Thomas Weise
In the example, why not have StateInternalsStateCleaner and TimeInternalsCleanupTimer depend on the context to get the current internals instead? On Thu, Apr 6, 2017 at 8:37 AM, JingsongLee wrote: > There is no suitable way to get the CurrentKey. > I think using StepContext.timerInternals() an

Re: StatefulDoFnRunner

2017-04-06 Thread JingsongLee
There is no suitable way to get the CurrentKey. I think using StepContext.timerInternals() and StepContext.stateInternals() is better. best, JingsongLee --From:Thomas Weise Time:2017 Apr 6 (Thu) 12:45To:dev Subject:StatefulDoFnRunn

StatefulDoFnRunner

2017-04-05 Thread Thomas Weise
Hi, While working on the support for splittable DoFn, I see a few cases where the DoFn runner classes slightly complicate reuse across elements (or make it a bit awkward to implement for the runner). StateInternalsStateCleaner and TimeInternalsCleanupTimer take xxxInternals instances. But since t