Re: Stateful DoFn, Global Window

2021-11-17 Thread gaurav mishra
Thanks for the clarifications Robert. On Tue, Nov 16, 2021 at 3:27 PM Robert Bradshaw wrote: > On Tue, Nov 16, 2021 at 3:00 PM gaurav mishra > wrote: > > > > Hi, > > I have a pipeline which looks like this - > > Input -> Convert_to_KV_pairsDoFn -> SomeStatefulDofn -> output > > As you can see t

Stateful DoFn, Global Window

2021-11-16 Thread gaurav mishra
Hi, I have a pipeline which looks like this - Input -> Convert_to_KV_pairsDoFn -> SomeStatefulDofn -> output As you can see there is no explicit "shuffle" transform here My understanding and observation so far has been that the SomeStatefulDofn will never be executed in parallel on two workers fo