Re: Beam on Flink runner not able to advance watermarks on a high load

Hi Sandeep, - dev@beam The watermark estimation itself should not be related to load. Can you please clarify, if  a) you are using any custom timestamp policy?  b) you see any backpressure in Flink's UI? Backpressure could - under some circumstances - cause del

Go SDK pubsubio.Write does not output anything running on Dataflow

Hi, Thank you and congratulations on graduating the Go SDK from experimental status- it's all very exciting. We're already successfully using it for several of our internal pipelines, but I've now run into an issue writing data using the pubsubio.Write transform and I am unable to figure it out

Re: Go SDK pubsubio.Write does not output anything running on Dataflow

While trying to reproduce the pipeline using the Python SDK I've noticed that the topic format is different for the write transform [1] compared to the read transform [2]. It seems it uses /topics// and /projects//topics/ respectively. This is also documented in the Python SDK documentation [3]

Stateful DoFn, Global Window

Hi, I have a pipeline which looks like this - Input -> Convert_to_KV_pairsDoFn -> SomeStatefulDofn -> output As you can see there is no explicit "shuffle" transform here My understanding and observation so far has been that the SomeStatefulDofn will never be executed in parallel on two workers fo