I've been looking at this a bit, and I think it will be tricky to figure
out how to get this working with BigQueryIO. The base problem is that Wait
tries to sequence things by window, as it's hard to come up with a
different definition of sequencing with unbounded inputs. However
BigQueryIO rewindo
Just opened this PR: https://github.com/apache/beam/pull/6055 to get
feedback ASAP. Basically what it does is return the job status in a
PCollection of BigQueryWriteResult objects
On Fri, Jul 20, 2018 at 11:57 PM Reuven Lax wrote:
> There already is a org.apache.beam.sdk.io.gcp.bigquery.WriteRes
There already is a org.apache.beam.sdk.io.gcp.bigquery.WriteResult class.
On Tue, Jul 17, 2018 at 9:44 AM Eugene Kirpichov
wrote:
> Hmm, I think this approach has some complications:
> - Using JobStatus makes it tied to using BigQuery batch load jobs, but the
> return type ought to be the same r
Hmm, I think this approach has some complications:
- Using JobStatus makes it tied to using BigQuery batch load jobs, but the
return type ought to be the same regardless of which method of writing is
used (including potential future BigQuery APIs - they are evolving), or how
many BigQuery load jobs
All good so far. I've been a bit side tracked but more or less I have the
idea of using the JobStatus as part of the collection so that not only the
completion is signaled, but also the result (success/failure) can be
accessed, how does it sound?
Regards
On Tue, Jul 17, 2018 at 3:07 AM Eugene Kir
Hi Carlos,
Any updates / roadblocks you hit?
On Tue, Jul 3, 2018 at 7:13 AM Eugene Kirpichov
wrote:
> Awesome!! Thanks for the heads up, very exciting, this is going to make a
> lot of people happy :)
>
> On Tue, Jul 3, 2018, 3:40 AM Carlos Alonso wrote:
>
>> + dev@beam.apache.org
>>
>> Just a
yes, i must say i have been waiting for this for over 6 months, it
would help a lot
chaim
On Tue, Jul 3, 2018 at 5:14 PM Eugene Kirpichov wrote:
>
> Awesome!! Thanks for the heads up, very exciting, this is going to make a lot
> of people happy :)
>
> On Tue, Jul 3, 2018, 3:40 AM Carlos Alonso w
Awesome!! Thanks for the heads up, very exciting, this is going to make a
lot of people happy :)
On Tue, Jul 3, 2018, 3:40 AM Carlos Alonso wrote:
> + dev@beam.apache.org
>
> Just a quick email to let you know that I'm starting developing this.
>
> On Fri, Apr 20, 2018 at 10:30 PM Eugene Kirpich
+ dev@beam.apache.org
Just a quick email to let you know that I'm starting developing this.
On Fri, Apr 20, 2018 at 10:30 PM Eugene Kirpichov
wrote:
> Hi Carlos,
>
> Thank you for expressing interest in taking this on! Let me give you a few
> pointers to start, and I'll be happy to help everywh
