Re: How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread P Singh
Hi, I have done this manually. though It seems the beam version issues, I am using 2.43. Is it compatible with flink 1.14 and python 3.10. Most of the threads indicating beam doesn't really support latest versions. On Sat, 4 Feb 2023 at 02:51, Robert Bradshaw via user wrote: > You should be abl

Re: How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread Robert Bradshaw via user
You should be able to omit the environment_type and environment_config variables and they will be populated automatically. For running locally, the flink_master parameter is not needed either (one will be started up automatically). On Fri, Feb 3, 2023 at 12:51 PM Talat Uyarer via user wrote: > >

Re: How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread Talat Uyarer via user
Hi, Do you use Flink operator or manually deployed session cluster ? Thanks On Fri, Feb 3, 2023, 4:32 AM P Singh wrote: > Hi Team, > > I have set up a flink cluster on GKE and am trying to submit a beam > pipeline with below options. I was able to run this on a local machine but > I don't unde

How to submit beam python pipeline to GKE flink cluster

2023-02-03 Thread P Singh
Hi Team, I have set up a flink cluster on GKE and am trying to submit a beam pipeline with below options. I was able to run this on a local machine but I don't understand what would be the environment_config? What should I do? what to put here instead of localhost:5 Please help. options = Pip