Re: Uneven TM Distribution of Flink on YARN

2023-09-06 Thread Lu Niu
, who is an expert on this. >>> Do you have any ideas on the phenomenon here? >>> >>> Best, >>> Zhanghao Chen >>> -- >>> *From:* Lu Niu >>> *Sent:* Tuesday, August 29, 2023 12:11:35 PM >>> *To:* Chen

Re: Uneven TM Distribution of Flink on YARN

2023-09-06 Thread Lu Niu
*From: *Lu Niu > *Date: *Thursday, September 7, 2023 at 12:17 AM > *To: *Geng Biao > *Cc: *Chen Zhanghao , Weihua Hu < > huweihua@gmail.com>, Kenan Kılıçtepe , user < > user@flink.apache.org> > *Subject: *Re: Uneven TM Distribution of Flink on YARN > > Hi, Than

Re: Uneven TM Distribution of Flink on YARN

2023-09-06 Thread Biao Geng
duling strategy, the final distribution of apps after some time is different. Best, Biao Geng From: Lu Niu Date: Thursday, September 7, 2023 at 12:17 AM To: Geng Biao Cc: Chen Zhanghao , Weihua Hu , Kenan Kılıçtepe , user Subject: Re: Uneven TM Distribution of Flink on YARN Hi, Thanks for all you

Re: Uneven TM Distribution of Flink on YARN

2023-09-06 Thread Lu Niu
t;> Zhanghao Chen >> ---------- >> *From:* Lu Niu >> *Sent:* Tuesday, August 29, 2023 12:11:35 PM >> *To:* Chen Zhanghao >> *Cc:* Kenan Kılıçtepe ; user > > >> *Subject:* Re: Uneven TM Distribution of Flink on YARN >> &g

Re: Uneven TM Distribution of Flink on YARN

2023-08-30 Thread Lu Niu
iOS <https://aka.ms/o0ukef> > -- > *发件人:* Chen Zhanghao > *发送时间:* Tuesday, August 29, 2023 12:14:53 PM > *收件人:* Lu Niu ; Weihua Hu > *抄送:* Kenan Kılıçtepe ; user > *主题:* Re: Uneven TM Distribution of Flink on YARN > > CCing @Weihua Hu , who is an expert on this.

Re: Uneven TM Distribution of Flink on YARN

2023-08-29 Thread Geng Biao
er 主题: Re: Uneven TM Distribution of Flink on YARN CCing @Weihua Hu<mailto:huweihua@gmail.com> , who is an expert on this. Do you have any ideas on the phenomenon here? Best, Zhanghao Chen From: Lu Niu Sent: Tuesday, August 29, 2023 12:11:35 PM To: C

Re: Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Chen Zhanghao
ct: Re: Uneven TM Distribution of Flink on YARN Thanks for your reply. The interesting fact is that we also managed spark on yarn. However. Only the flink cluster is having the issue. I am wondering whether there is a difference in the implementation on flink side. Best Lu On Mon, Aug 28, 2023 at 8

Re: Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Lu Niu
ts Standalone mode Flink > clusters, and does not take effect on a Flink cluster on YARN. > > Best, > Zhanghao Chen > -- > *发件人:* Lu Niu > *发送时间:* 2023年8月29日 4:30 > *收件人:* Kenan Kılıçtepe > *抄送:* user > *主题:* Re: Uneven TM Distribution of Flink

回复: Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Chen Zhanghao
mode Flink clusters, and does not take effect on a Flink cluster on YARN. Best, Zhanghao Chen 发件人: Lu Niu 发送时间: 2023年8月29日 4:30 收件人: Kenan Kılıçtepe 抄送: user 主题: Re: Uneven TM Distribution of Flink on YARN Thanks for the reply. We've already set cluster.e

Re: Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Lu Niu
Thanks for the reply. We've already set cluster.evenly-spread-out-slots = true Best Lu On Mon, Aug 28, 2023 at 1:23 PM Kenan Kılıçtepe wrote: > Have you checked config param cluster.evenly-spread-out-slots ? > > > On Mon, Aug 28, 2023 at 10:31 PM Lu Niu wrote: > >> Hi, Flink users >> >> We hav

Re: Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Kenan Kılıçtepe
Have you checked config param cluster.evenly-spread-out-slots ? On Mon, Aug 28, 2023 at 10:31 PM Lu Niu wrote: > Hi, Flink users > > We have recently observed that the allocation of Flink TaskManagers in our > YARN cluster is not evenly distributed. We would like to hear your thoughts > on this

Uneven TM Distribution of Flink on YARN

2023-08-28 Thread Lu Niu
Hi, Flink users We have recently observed that the allocation of Flink TaskManagers in our YARN cluster is not evenly distributed. We would like to hear your thoughts on this matter. 1. Our setup includes Flink version 1.15.1 and Hadoop 2.10.0. 2. The uneven distribution is that out of a 370-node