Re: Beam 2.28.0 objectReuse and fasterCopy for FlinkPipelineOption

2021-04-09 Thread Jan Lukavský
Hi Eleanore, the --fasterCopy option disables clone between operators (see [1]). It should be safe to use it, unless your pipeline outputs an object and later modifies the same instance. This is generally not supported by the Beam model and is considered to be an user error. FlinkRunner histo

Side windows and late inputs

2021-04-09 Thread Raman Gupta
I have a topology that looks like this: input -> 1hr window -> view | -> 5m window -> dofn(view) IOW -- an input is windowed into both 1 hour windows, as well as 5 minute windows. The 1 hour window data is collected into a view. Meanwhile, the input is also windowed with 5 minute win