[Question]: Side input got stuck

2022-08-02 Thread P Singh
Hi There, I am implementing a beam pipeline,which is reading events from pub-sub and using sliding windowing and then combinegloblally. The thing is that I am not able to test my pipeline on local machine. below is the code sample. I am not able to feed event_out to event_fi my pipeline gets froze

Re: [Question]: Side input got stuck

2022-08-02 Thread P Singh
t 23:03, Mátyás Manninger via user wrote: > I had similar issues with the local runner, I deployed the code to > Dataflow and it worked. Try deploying it and see if that helps. > > On 2022. Aug 2., Tue at 16:31, P Singh > wrote: > >> Hi There, >> >> I am impl

Re: [Question]: Side input got stuck

2022-08-02 Thread P Singh
many times per window? On Tue, 2 Aug 2022 at 23:23, P Singh wrote: > Hi Mátyás, > > Ohh okay I can try that but something throws this error too if there's no > data in the pub-sub topic. > > window.WindowFn.AssignContext(source_window.max_timestamp([-1] > Attribute

Re: [ANNOUNCE] Apache Beam 2.41.0 Released

2022-08-26 Thread P Singh
Hi Team, Looking forward to trying and testing the new version, It's always fascinating. On Fri, 26 Aug 2022 at 03:45, Pablo Estrada via user wrote: > Thank you Kiley! > > On Thu, Aug 25, 2022 at 10:55 AM Kiley Sok wrote: > >> The Apache Beam team is pleased to announce the release of version

Re: [Question] Beam SQL failed with NPE

2022-08-30 Thread P Singh
Hi, Use apache-beam==2.37.0 On Wed, 31 Aug 2022 at 00:02, Brian Hulette via user wrote: > Hi Zheng, > Could you share a minimal example that reproduces the issue? > Also, have you tried using Beam >2.35.0? I'm curious if this happens in > the 2.41.0 release as well. > > Brian > > On Tue, Aug 30

Re: [question] Good Course to learn beam

2022-08-30 Thread P Singh
Hi, Yeah I agree this book covers pretty much what you need as beginner, though I would suggest building stuff along reading. On Wed, 31 Aug 2022 at 03:08, Adeel wrote: > Hi, > > The packt ebook is a lot better, guide yourself practically as you > read the book. Plus, it is from beam committer.

How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread P Singh
Hi Team, I have set up a flink cluster on GKE and am trying to submit a beam pipeline with below options. I was able to run this on a local machine but I don't understand what would be the environment_config? What should I do? what to put here instead of localhost:5 Please help. options = Pip

Re: How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread P Singh
er > wrote: > > > > Hi, > > > > Do you use Flink operator or manually deployed session cluster ? > > > > Thanks > > > > On Fri, Feb 3, 2023, 4:32 AM P Singh > wrote: > >> > >> Hi Team, > >> > >> I have set up