Re: Parameters to Control Intra-node Parallelism

2016-07-02 Thread Saliya Ekanayake
Thank you, yes, it can be done externally, if not supported within Flink. So the way to spawn multiple task managers would be to list the same slave machines N times as necessary in the slaves file? On Sat, Jul 2, 2016 at 11:22 AM, Ufuk Celebi wrote: > No, not inside of Flink. That sounds like

Re: Parameters to Control Intra-node Parallelism

2016-07-02 Thread Ufuk Celebi
No, not inside of Flink. That sounds like something like the OS or resource manager should handle. On Sat, Jul 2, 2016 at 5:12 PM, Saliya Ekanayake wrote: > That's great, so is there support to pin task managers to sockets as well? > > On Sat, Jul 2, 2016 at 11:08 AM, Ufuk Celebi wrote: >> >> Re

Re: Parameters to Control Intra-node Parallelism

2016-07-02 Thread Saliya Ekanayake
That's great, so is there support to pin task managers to sockets as well? On Sat, Jul 2, 2016 at 11:08 AM, Ufuk Celebi wrote: > Regarding 2) if you don't manually configure something else, that > should happen always. > > Yes, you can run more than one task manager per node depending on the > p

Re: Parameters to Control Intra-node Parallelism

2016-07-02 Thread Ufuk Celebi
Regarding 2) if you don't manually configure something else, that should happen always. Yes, you can run more than one task manager per node depending on the process isolation you want. Within a task manager, there are multiple threads for each slot. For example, if you have 2 task managers with 2