ained" mode but there may need to be a middle ground
> > > where tasks are "coarse-grained" i.e. run multiple operators and once
> idle
> > > for a certain amount of time, they are reaped/killed-off by the
> > > jobManager/integration-framework.
> > >
&g
o one module which utilises
> > another module (Yarn or Mesos) that takes such a request and satisfies the
> > needs of the former. This idea is sort of inspired from the way this
> > separation exists in apache spark and seems to work out well.
> >
> > * I don
a general scheduler) that
> > says "get me a slot meeting constraints X" into one module which utilises
> > another module (Yarn or Mesos) that takes such a request and satisfies
> the
> > needs of the former. This idea is sort of inspired from the way this
> > separation exists in apache spark and seems to work out well.
> >
> > * I don't know the codebase well enough to say where these things go
> based
> > on my reading of the overall architecture of the system, there is nothing
> > that can't be satisfied by the flink-runtime and it *should* not need any
> > detailed access to the execution plan. I'll defer this to someone who
> knows
> > the internals better.
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/add-some-new-api-to-the-scheduler-in-the-job-manager-tp7153p7448.html
> > Sent from the Apache Flink Mailing List archive. mailing list archive at
> > Nabble.com.
> >
>
> * I don't know the codebase well enough to say where these things go based
> on my reading of the overall architecture of the system, there is nothing
> that can't be satisfied by the flink-runtime and it *should* not need any
> detailed access to the execution plan. I'll defer this to someone who knows
> the internals better.
>
>
>
> --
> View this message in context:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/add-some-new-api-to-the-scheduler-in-the-job-manager-tp7153p7448.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at
> Nabble.com.
>
based
on my reading of the overall architecture of the system, there is nothing
that can't be satisfied by the flink-runtime and it *should* not need any
detailed access to the execution plan. I'll defer this to someone who knows
the internals better.
--
View this message in contex
t;
>>
>> > From: rmetz...@apache.org
>> > Date: Mon, 27 Jul 2015 14:01:11 +0200
>> > Subject: Re: add some new api to the scheduler in the job manager
>> > To: dev@flink.apache.org
>> >
>> > Hi MaGuoWei,
>> >
>> > would
..@apache.org
> > Date: Mon, 27 Jul 2015 14:01:11 +0200
> > Subject: Re: add some new api to the scheduler in the job manager
> > To: dev@flink.apache.org
> >
> > Hi MaGuoWei,
> >
> > would you like to have done automatically by Flink or based on some user
.
> From: rmetz...@apache.org
> Date: Mon, 27 Jul 2015 14:01:11 +0200
> Subject: Re: add some new api to the scheduler in the job manager
> To: dev@flink.apache.org
>
> Hi MaGuoWei,
>
> would you like to have done automatically by Flink or based on some user
> inpu
..@apache.org
> > > Date: Mon, 27 Jul 2015 05:14:00 +
> > > Subject: Re: add some new api to the scheduler in the job manager
> > > To: dev@flink.apache.org
> > >
> > > Hi,
> > > I think for more details on giving back resources of a running cl
uoWei wrote:
> It is great! Is there any document? I am very interested in this.
> thanks
>
>
> > From: aljos...@apache.org
> > Date: Mon, 27 Jul 2015 05:14:00 +0000
> > Subject: Re: add some new api to the scheduler in the job manager
> > To: dev@flink.apache.org
It is great! Is there any document? I am very interested in this.
thanks
> From: aljos...@apache.org
> Date: Mon, 27 Jul 2015 05:14:00 +
> Subject: Re: add some new api to the scheduler in the job manager
> To: dev@flink.apache.org
>
> Hi,
> I think for more details on
Hi,
I think for more details on giving back resources of a running cluster we
have to wait for Robert's opinion. In the mean time, you can also just run
a single job that will bring up some yarn containers and then release them
afterward using this:
https://ci.apache.org/projects/flink/flink-docs-r
hi guysFlink must take over all the resources all the time. That wastes
resources sometime especially in the sharing cluster.For example when using
Flink on Yarn the resource can't be returned back to RM even if no graph is
running.So I want to know: Is it possible to add some apis in the sche
13 matches
Mail list logo
