Our main use cases are mesos, maybe we can start with mesos support.
On Wed, Jun 6, 2018 at 5:00 PM Stephan Ewen wrote:
> The FLIP-6 design was specifically such that it allows for separation of
> Dispatcher, ResourceManager, and JobManagers.
> So that could be another extension at some point.
>
The FLIP-6 design was specifically such that it allows for separation of
Dispatcher, ResourceManager, and JobManagers.
So that could be another extension at some point.
It should be conceptually rather simple, the dispatcher creates per job a
new container launch context with the "JobManagerRunner
That's really great! I'll help to contribute to the process.
On Wed, Jun 6, 2018 at 3:17 PM Till Rohrmann wrote:
> Hi Renjie,
>
> there is already an issue for introducing further scheduling constraints
> (e.g. tags) to achieve TM isolation when using the session mode [1]. What
> it does not cov
Hi Renjie,
there is already an issue for introducing further scheduling constraints
(e.g. tags) to achieve TM isolation when using the session mode [1]. What
it does not cover is the isolation of the JMs which need to be executed in
their own processes. At the moment they share the same process wi
Hi, Stephan:
Yes that's what I mean. In fact the most import thing is to share the
dispatcher so that we can have *a centralized gateway for flink job
management and submission. The problem with per job cluster is that we
can't have a centralized gateway.*
I didn't realize that job manager also n
Hi Renjie,
When you suggest to have TaskManager isolation in session mode, do you mean
to have a shared JobManager / Dispatcher, but job-specific TaskManagers?
If this mainly to reduce the overhead of the per-job JobManager?
One assumption so far was that if TaskManager isolation is required,
Job
Hi, Till:
1. Does the community has any plan to add task manager isolation into
the session mode?
2. Is there any issues to track this feature? I want to help contribute.
3. Thanks for the knowledge but it can't help if task manager isolation
is not present.
On Tue, Jun 5, 2018 a
Hi Renjie,
1) you're right that the Flink session mode does not give you proper job
isolation. It is the same as with Flink 1.4 session mode. If this is a
strong requirement for you, then I recommend using the per job mode.
2) At the moment it is also not possible to define per job resource
requi
