Occasional RPC DEADLINE_EXCEEDED with PubSub source on Python/DirectRunner

2020-06-06 Thread Pradip Thachile
Hi, I'm testing a pipeline using Python Beam and the DirectRunner that reads from a Pub/Sub subscription however after "some time" (i.e. I don't quite see a predictable pattern yet) I get a bundle failure exception that flags a failure with a gRPC call to Pub/Sub: ERROR:apache_beam.runners.dir

Re: Berlin Buzzwords - Vouchers

2020-06-06 Thread Obia Ugochukwu
Hi Brittany, I would be interested in getting a ticket code for the Berlin Buzzwords. Regards. On Fri, Jun 5, 2020 at 10:11 PM Brittany Hermann wrote: > Hi folks, > > As a sponsor for Berlin Buzzwords, we were given some ticket codes. If any > of you are interested in receiving one of the code

Writing pipeline output to google sheet in google drive

2020-06-06 Thread OrielResearch Eila Arich-Landkof
Hello, Is it possible to have the pipeline sink to a google sheet within a specific google drive directory. Something like that: p = beam.Pipeline(options=options) (p | 'Step 1: read file ' >> beam.io.ReadFromText(path/to/file) | 'Step 2: process data ' >> beam.ParDo(get_daata(l])) | 's