答复:答复:thread model issue in TaskManager

2015-08-05 Thread wangzhijiang999
.  --发件人:Stephan Ewen 发送时间:2015年8月3日(星期一) 22:17收件人:user ,wangzhijiang999 主 题:Re: 答复:thread model issue in TaskManager  - Communication to the TaskManager, or directly to the JobManager  - Network stack for shuffles to exchange data with other processes (as exchanges go streaming and through

Re: 答复:thread model issue in TaskManager

2015-08-03 Thread Stephan Ewen
> > 发件人:Stephan Ewen > > 发送时间:2015年8月3日(星期一) 00:36 > > 收件人:user > > 抄 送:wangzhijiang999 > > 主 题:Re: thread model issue in TaskManager > > > > Here are some additional things you can do: > > > - For isolation between paralle

答复:thread model issue in TaskManager

2015-08-02 Thread wangzhijiang999
Hi Stephan,Fabian        Thank you for your reply!  I will run the flink on yarn actually . It is feasible to isolate different tasks in one job by starting new yarn session. And it means every job will have a yarn seesion, and one taskManager just has one slot. If I want to run all jobs in one

Re: thread model issue in TaskManager

2015-08-02 Thread Stephan Ewen
As Fabian suggested, YARN is a good way to go for isolation (it actually isolates more than a JVM, which is very nice). Here are some additional things you can do: - For isolation between parallel tasks (within a job), start your YARN job such that each TaskManager has one slot, and start many

Re: thread model issue in TaskManager

2015-07-30 Thread Fabian Hueske
Hi, it is currently not possible to isolate tasks that consume a lot of JVM heap memory and schedule them to a specific slot (or TaskManager). If you operate in a YARN setup, you can isolate different jobs from each other by starting a new YARN session for each job, but tasks within the same job c

thread model issue in TaskManager

2015-07-29 Thread wangzhijiang999
As I know, flink uses thread model in TaskManager, that means one taskmanager process may run many different operator threads,and these threads will compete the memory of the process. I know that flink has memoryManage component in each taskManager, and it will control the localBufferPool of Inp