Re: How to auto scale yarn session job

2023-03-20 Thread Si-li Liu
Sorry, it seems related our own bug, after fixed the defect it can scale automatically. Thanks for your reply. Weihua Hu 于2023年3月21日周二 12:02写道: > Hi, > > Yarn session clusters should auto allocate new task managers if slots are > not enough. > And the new submitting should not affect running jo

Re: How to auto scale yarn session job

2023-03-20 Thread Weihua Hu
Hi, Yarn session clusters should auto allocate new task managers if slots are not enough. And the new submitting should not affect running jobs. Could you provide the failure log? Best, Weihua On Tue, Mar 21, 2023 at 11:57 AM Si-li Liu wrote: > I use this command to launch a flink yarn sessi

How to auto scale yarn session job

2023-03-20 Thread Si-li Liu
I use this command to launch a flink yarn session. yarn-session.sh -s 6 -jm 2048 -tm 4096 -nm sql-client-session -m yarn-cluster -d And all my flink sql job has 2 parallelism, and I found that my yarn session can only have 3 pipelines. If my session doesn't have free slot, submit to this session