Re: Extremely Slow DirectRunner

Any further thoughts here? Or tips on profiling Beam DirectRunner? Thanks, Evan On Wed, May 12, 2021 at 6:22 PM Evan Galpin wrote: > Ok gotcha. In my tests, all sdk versions 2.25.0 and higher exhibit slow > behaviour regardless of use_deprecated_reads. Not sure if that points to > something dif

Re: [EXT] Re: [EXT] Re: [EXT] Re: [EXT] Re: [EXT] Re: Beam Dataframe - sort and grouping

Thank you for the clarification! Is there a way to control the number of shards, i.e. the bundle? I know in pure Beam IO connectors, we have num_shards supported, for example, WriteToParquet