Re: Submit Flink 1.11 job from java

2020-08-07 Thread David Anderson
Flavio, Have you looked at application mode [1] [2] [3], added in 1.11? It offers at least some of what you are looking for -- the application jar and its dependencies can be pre-uploaded to HDFS, and the main() method runs on the job manager, so none of the classes have to be loaded in the client

Re: Submit Flink 1.11 job from java

2020-08-07 Thread Flavio Pompermaier
The problem with env.executeAsync is that I need to load the job classes on the client side and this is something I'd like to avoid because it's a source of problems. I'd like to tell Flink to run a jar that is available somewhere (on the flink instances or on the blob server or on a network filesy

Re: Submit Flink 1.11 job from java

2020-08-06 Thread godfrey he
hi Flavio, Maybe you can try env.executeAsync method, which just submits the job and returns a JobClient. Best, Godfrey Flavio Pompermaier 于2020年8月6日周四 下午9:45写道: > Hi to all, > in my current job server I submit jobs to the cluster setting up an SSH > session with the JobManager host and running

Submit Flink 1.11 job from java

2020-08-06 Thread Flavio Pompermaier
Hi to all, in my current job server I submit jobs to the cluster setting up an SSH session with the JobManager host and running the bin/flink run command remotely (since the jar is put in the flink-web-upload directory). Unfortunately, this approach makes very difficult to caputre all exceptions th