Re: Resource isolation in flink among multiple jobs

2016-08-29 Thread Robert Metzger
Hi, this is the JIRA for mesos support: https://issues.apache.org/jira/browse/FLINK-1984 The first part of it has been merged today: https://github.com/apache/flink/pull/2315 So its likely going to be fully functional for the 1.2 release. You can run multiple TaskManager instances per machine, an

Re: Resource isolation in flink among multiple jobs

2016-08-29 Thread Abhishek Agarwal
Hi Robert, Thanks for your reply. Few follow up questions - Is there any timeline for mesos support? In standalone installations, how about having one Task slot per TaskManager and multiple TaskManager instances per machine? On Mon, Aug 29, 2016 at 7:13 PM, Robert Metzger wrote: > Hi, > > for i

Re: Resource isolation in flink among multiple jobs

2016-08-29 Thread Robert Metzger
Hi, for isolation, we recommend using YARN, or soon Mesos. For standalone installations, you'll need to manually set up multiple independent Flink clusters within one physical cluster if you want them to be isolated. Regards, Robert On Mon, Aug 29, 2016 at 1:41 PM, Abhishek Agarwal wrote: >

Resource isolation in flink among multiple jobs

2016-08-29 Thread Abhishek Agarwal
In a standalone flink cluster, the TaskManager has multiple slots and can run different applications in different slot but still within same JVM. Given that, two applications are running under same process, how is the cpu and memory isolation achieved? Are there any recommendations to achieve isol