Re: YARN reserved container prevents new Flink TMs

2019-01-18 Thread suraj7
Hi, Sharing new findings. The issue I have mentioned above seems to be happening only with the latest version of EMR(emr-5.20.0, hadoop: Amazon 2.8.5, Flink: 1.6.2) and it is reproducible with our setup every time. I have verified the same setup working and scaling without any issues on an older

YARN reserved container prevents new Flink TMs

2019-01-17 Thread suraj7
Hi, I am using Amazon EMR(emr-5.20.0, hadoop: Amazon 2.8.5, Flink: 1.6.2) to run Flink Cluster on YARN. My setup consists of m4.large instances for 1 master and 3 core nodes. I have Flink Cluster running on YARN with the command: flink-yarn-session -tm 5120 -s 3 -jm 1024. This setup should ideall