Hi, usually it doesn't make sense to run multiple task managers on a single machine to get more slots. Your machine has only 4 CPU cores, so you are just putting a lot of pressure on the cpu scheduler..
On Thu, Mar 31, 2016 at 7:16 PM, Shinhyung Yang <shinhyung.y...@gmail.com> wrote: > Thank you for replying! > > I am trying to do this on a single machine in fact. Since it has 64 > cores, it would be interesting to look at the performance in that > regard. > > > How many machines are you using for this? > > > > The fact that you are giving 64 slots to each TaskManager means that a > > single TaskManager may end up executing all 64 pipelines. That would > heavily > > overload that TaskManager and cause heavy degradation. > > Does it make sense if I run multiple TaskManagers on a single machine > if 64 slots are too many for a TaskManager? > > > If, for example, you use 16 machines, then give each machine 4 task slots > > (total of 64 slots across all machines) > > That way, the final run (parallelism 64) will be guaranteed to be spread > > across all machines. > > My intention for the experiment at the moment is to try to scale the > application up on a single machine to its maximum before moving on to > run the experiment on multiple machines. > > Thank you again! > With best regards, > Shinhyung Yang >
