Which flink version do you use ?
Garrett Barton <garrett.bar...@gmail.com>于2018年7月11日周三 上午1:09写道： > Hey all, > I am running flink in batch mode on yarn with independant jobs creating > their own clusters. > > I have a flow defined that scales parallelism based on input size (to keep > overall processing time somewhat constant). Right now the flow initializes > with around ~22k tasks for a flow that sets parallelism to 1600 for a few > portions. At the cli I launch flink with: > > flink run -m yarn-cluster -yn 200 -ys 4 -yqu test -c bla.. jar > > My expectation is that this would create a max cluster of 800 slots from > 200 task managers. When I run with the input that scales to parallel 1600 > I end up with over 2832 slots and 708 task managers. What looks like its > doing is flink is consuming the entirety of the available yarn queue its > assigned to (happened to get ~75% of the queue that last run). > > Does -yn act as a suggestion rather than a limit? >
