How many task managers to launch for a job?

Hi, I am redesigning the scheduler of the JobManager to place tasks of a job across TaskManagers accroding to a scheduling policy. I am reading the Flip-6 proposal and found that the common case is "one TaskManager launchs one slot", and "one Flink cluster serves one job". But I did not find how

Re: How many task managers to launch for a job?

che.org/projects/flink/flink-docs-release-1.8/concepts/runtime.html#task-slots-and-resources> > . > > Thank you~ > > Xintong Song > > > > On Thu, May 23, 2019 at 11:49 PM black chase > wrote: > >> >> Hi, >> >> I am redesigning the scheduler of

What policies does YARN/Mesos use to distribute TaskManagers across physical machines?

Hi all, I had a question when I was reading Flip-6. When submitting a job, YARN/Mesos ResourceManager needs to spawn, let's say, 5 TaskManagers for the job. Suppose we have 4 physical machines, how would ResourceManager place the 5 TaskManagers across the 4 machines? Does it just follow a Round-Rob

Re: How many task managers to launch for a job?

any requirements on how the > TaskManagers are distributed across physical machines. So I think it really > depends on the scheduling policy of the Mesos cluster. I'm not an expert on > Mesos, so correct me if I was wrong. > > Thank you~ > > Xintong Song > > > > On

Re: How many task managers to launch for a job?

For the sake of simplicity, the following talks about “slots”, but one can think simply of “TaskManager” instead, for the common case of a one-slot TaskManager.". It seems the common practice is to have one slot for one taskmanager. Best, Chase On Fri, May 24, 2019 at 11:28 AM black chase wrot

Re: How many task managers to launch for a job?

e of each TaskManager also depends on the job's > resource needs and your environment. So having one slot for each > TaskManager would be a simple choice because it avoids tuning these two > relevant factors at the same time. > > Thank you~ > > Xintong Song > > > &