Re: Normal Spark Streaming vs Streaming on Beam with Spark Runner

Thanks Lukasz. It was really helpful to understand. Regards, Chandan On Thu, May 17, 2018 at 8:26 PM, Lukasz Cwik wrote: > > > On Wed, May 16, 2018 at 10:46 PM chandan prakash < > chandanbaran...@gmail.com> wrote: > >> Thanks Ismaël. >> Your answers were quite useful for a novice user. >> I gu

Doubled trigger frequency writing to BQ

Hi, I am using BigQueryIO from Apache Beam 2.4.0 to write data to BQ (Write.Method.FILE_LOADS). My volumes per second will be pretty low -- 200 - 2000 rows per second. What I'm finding is that data often takes twice as long to be inserted into BQ as the configured trigger frequency. My code is s