Re: side input terribly slow when using Flink runner

Hi Stefan, could you verify what is the coder you use for the PCollection, which you materialize as side-input? I'm not sure from the flame-graph itself, but could it be SerializableCoder?  Jan On 10/12/21 12:23 PM, Stefan Wachter wrote: Hi, I have a pipeline where are PCollection is fed a

Re: Perf issue with Beam on spark (spark runner)

Robert, Do you have any numbers by chance regarding this optimisation? Alexey > On 5 Oct 2021, at 00:27, Robert Bradshaw wrote: > > https://github.com/apache/beam/pull/15637 > might help some. > > On Thu, Sep 9, 2021 at 5:21 PM Tao Li

Reading from a BigQuery view in another project

Hi, We need to read from a BigQuery view from a data provider project. I run into two issues: 1. Is it possible to use beam.io.ReadFromBigQuery with a view (not a table) 2. In order to read, the user needs bigquery.jobs.create permissions. But we want to create the job in our project, not in th

Re: Reading from a BigQuery view in another project

The problem with that solution is that I would effectively use the vendor resources every time I read the data (and they would incur GCP costs). I know that GCP can support this such that I access the data in the vendor project, but the execution happens in my project. Thanks Mark On Tue, O

Re: Beam/Python to BigTable

Dear Community, Glad to get your support here ! Issue: empty BigTable when using the Python/Beam connector. Thanks ! Le dim. 10 oct. 2021 à 14:34, Pierre Oberholzer a écrit : > Thanks Israel, this helped. No error anymore, but the table remains empty > with this code >