Hi, I have done this manually. though It seems the beam version issues, I am using 2.43. Is it compatible with flink 1.14 and python 3.10. Most of the threads indicating beam doesn't really support latest versions.
On Sat, 4 Feb 2023 at 02:51, Robert Bradshaw via user <user@beam.apache.org> wrote: > You should be able to omit the environment_type and environment_config > variables and they will be populated automatically. For running > locally, the flink_master parameter is not needed either (one will be > started up automatically). > > On Fri, Feb 3, 2023 at 12:51 PM Talat Uyarer via user > <user@beam.apache.org> wrote: > > > > Hi, > > > > Do you use Flink operator or manually deployed session cluster ? > > > > Thanks > > > > On Fri, Feb 3, 2023, 4:32 AM P Singh <bigdatadevelop...@gmail.com> > wrote: > >> > >> Hi Team, > >> > >> I have set up a flink cluster on GKE and am trying to submit a beam > pipeline with below options. I was able to run this on a local machine but > I don't understand what would be the environment_config? What should I do? > what to put here instead of localhost:50000 > >> > >> Please help. > >> options = PipelineOptions([ > >> "--runner=FlinkRunner", > >> "--flink_version=1.14", > >> "--flink_master=localhost:8081", > >> "--environment_type=EXTERNAL", #EXTERNAL > >> "--environment_config=localhost:50000", > >> ]) >
