Implementing a custom I/O Connector

Hi Everyone, I've been working on implementing a Google Sheets IO source for my pipeline. I've tried this example along with this blog . I have an example here on colab

Re: Implementing a custom I/O Connector

ath via user < user@beam.apache.org> wrote: > Do you have the full stacktrace ? > Also, what does the Read() transform in the example entail ? > > Thanks, > Cham > > On Thu, Jul 14, 2022 at 7:39 AM Damian Akpan > wrote: > >> Hi Everyone, >> >> I&#x

Re: Implementing a custom I/O Connector

gt; p | beam.Create([source_description]) | beam.ParDo(CountFn(10)) > > If the 'source_description' element is trivial (or gets ignored in the > source), you can replace 'beam.Create([source_description])' with > 'beam.Impulse()'. > > Thanks, > Cham &g

Re: [Dataflow][Python] Guidance on HTTP ingestion on Dataflow

Provided you have all the resources ids ahead of fetching, Beam will spread the fetches to its workers. It will still fetch synchronously but within that worker. On Tue, Jul 19, 2022 at 5:40 PM Shree Tanna wrote: > Hi all, > > I'm planning to use Apache beam to extract and load part of the ETL >