Re: [Discussion] Move cluster creation functions from flink-cli.sh to other scripts

2018-08-21 Thread Till Rohrmann
How would that look in detail given that I am a user who wants to start a job cluster? On Tue, Aug 21, 2018 at 12:09 PM Renjie Liu wrote: > But it's also dangerous if user need to submit job via the same cluster > configuration. I mean we can use another script for starting job cluster, > so tha

Re: [Discussion] Move cluster creation functions from flink-cli.sh to other scripts

2018-08-21 Thread Renjie Liu
But it's also dangerous if user need to submit job via the same cluster configuration. I mean we can use another script for starting job cluster, so that cluster administrators can separate environments. On Tue, Aug 21, 2018 at 5:53 PM Till Rohrmann wrote: > Hi Renjie, > > do you mean to not supp

Re: [Discussion] Move cluster creation functions from flink-cli.sh to other scripts

2018-08-21 Thread Till Rohrmann
Hi Renjie, do you mean to not support deploying Flink clusters via `FLINK_HOME/bin/flink -m yarn-cluster`? I think starting a dedicated job cluster for a given job that way is quite useful. Cheers, Till On Thu, Aug 9, 2018 at 4:33 AM Renjie Liu wrote: > Hi: > Flink 1.5.0 brings new deployment

[Discussion] Move cluster creation functions from flink-cli.sh to other scripts

2018-08-08 Thread Renjie Liu
Hi: Flink 1.5.0 brings new deployment mode and we are able to provid a single cluster for our jobs which manages resources elastically. However, this also brings challenges for our cluster management since users can submit jobs or create yarn cluster in the same flink-cli.sh. Here is the motivation