Re: Writing pipeline output to google sheet in google drive

2020-06-08 Thread Austin Bennett
@OrielResearch Eila Arich-Landkof Depending on your needs, I wonder about establishing a sheet (or sheets, as needed) that has a BQ connector for the datasource of it. If you use Dataflow to write/create a BQ table, that would then hydrate the sheet (not sure the ordering -- maybe you'd need to

Re: Writing pipeline output to google sheet in google drive

2020-06-08 Thread Luke Cwik
It doesn't look like BigQuery supports exporting to Google sheet[1], maybe you can invoke this BQ connector directly by adding a transform that follows the BQ sink. 1: https://cloud.google.com/bigquery/docs/exporting-data#export_limitations On Sat, Jun 6, 2020 at 8:31 PM OrielResearch Eila Arich-

Writing pipeline output to google sheet in google drive

2020-06-06 Thread OrielResearch Eila Arich-Landkof
Hello, Is it possible to have the pipeline sink to a google sheet within a specific google drive directory. Something like that: p = beam.Pipeline(options=options) (p | 'Step 1: read file ' >> beam.io.ReadFromText(path/to/file) | 'Step 2: process data ' >> beam.ParDo(get_daata(l])) | 's