Is there any job board for Beam with Java?

Hi All, Sorry to spam. I am writing to check if there is any job board where we can post or look for open positions/ jobs in Apache Beam? Thanks Deepak

Advanced Java 1.8+ and Beam mentors

Hi I am looking for some advanced level of mentoring in Beam using Java 1.8+. Are there any resources available or if someone does it on professional development level? Please let me know. Thanks Deepak

How to debug dataflow locally

Hi All, I am trying to come up with a framework that would help debugging dataflow job end to end locally. Dataflow can work with 100s of source and sinks. Is there a framework already to setup these sources and sinks locally e.g. if my dataflow reads from BQ and inserts to Bigtable. Please let me

Design patterns while using Beam

Hi All I guess we need to put some examples in the documentation around best coding practises , concurrency , non blocking IO and design patterns while writing Apache Beam pipelines. Is there any such guide available ? E.g. when there are lot of options to be used in the pipeline , BuilderPattern s

Re: Scheduling dataflow pipelines

We have used composer (airlfow) successfully to schedule Dataflow jobs. Please let me know if you would need details around it. Thanks Deepak On Sun, Apr 5, 2020 at 7:56 PM Joshua B. Harrison wrote: > Hi Marco, > > I've ended using a VM running Luigi to schedule jobs. I use the data flow > Pyth

Ways to learn Beam in detail

Hi There, Can anyone suggest the best way to get started on Beam APIs , so i can start contributing to the codebase? Thanks Deepak

Freelancing work

Hi All, Sorry for spamming. If anyone is looking or knows someone who might be looking for freelancers in Apache Beam,please let me know. Thanks.

Re: Allyship workshops for open source contributors

+1 On Tue, Jun 15, 2021 at 12:31 AM Aizhamal Nurmamat kyzy wrote: > Thank you all! Based on the feedback, I will set up a session for a couple > open source groups. Will share more details soon. Stay tuned. > > On Mon, Jun 7, 2021 at 4:42 PM Kenneth Knowles wrote: > >> Yes please! >> >> On Thu,

Re: Firing a dataflow job using REST API

Hi Rajnil, Did you look at the gcloud documentation? https://cloud.google.com/dataflow/docs/reference/rest Thanks Deepak On Mon, Feb 14, 2022 at 3:53 PM Rajnil Guha wrote: > Hi Beam Community, > > I am trying to run a Dataflow pipeline using the REST API. Currently I see > an example for doing

Re: [Question] AWS Credentials Serializability

It seems there is problem with the fat/Uber jar . It might have multiple versions of the classs being overwritten with duplicates . Check this link : https://stackoverflow.com/questions/57039951/beam-failed-to-serialize-and-deserialize-property-awscredentialsprovider On Sat, 23 Sep 2023 at 9:55