Hi Ankur,
I am not aware of any up-to-date papers about the internals of Flink, but
the links on this wiki page:
https://cwiki.apache.org/confluence/display/FLINK/Flink+Internals contain a
lot of helpful material.
I'm very happy to see that you are interested in contributing and
maintaining the f
Hi Robert,
I agree with everything that you and Stephan are saying. I haven't looked
into the flink codebase and the papers/design docs to comment at a finer
level so maybe that's the first piece of homework that I need to do ( need
pointers for that).
And, yes I would definitely be interested i
Hi,
one thing to also keep in mind here is that TaskManagers might have to keep
the intermediate data even after a job is finished. For example, if the
user has an interactive scala-shell session where they are exploring same
data and transforming it in several steps as they go.
Cheers,
Aljoscha
Hi,
I'm sorry for the late reply, I'm still working through a long email
backlog from a one week vacation ;)
Thank you for the long reply in this thread. Your observations are very
good.
I think we should indeed work towards a more fine grained "intra-job"
elasticity. Let me comment on some of you
Hi Stephan / others interested,
I have been working on the flink-mesos integration and there are definitely
some thoughts that I would like to share some thoughts about the
commonalities with the flink-yarn integration.
* Both flink-mesos and flink-yarn integration as they stand today can be
con
t;
>>
>> > From: rmetz...@apache.org
>> > Date: Mon, 27 Jul 2015 14:01:11 +0200
>> > Subject: Re: add some new api to the scheduler in the job manager
>> > To: dev@flink.apache.org
>> >
>> > Hi MaGuoWei,
>> >
>> > would
..@apache.org
> > Date: Mon, 27 Jul 2015 14:01:11 +0200
> > Subject: Re: add some new api to the scheduler in the job manager
> > To: dev@flink.apache.org
> >
> > Hi MaGuoWei,
> >
> > would you like to have done automatically by Flink or based on some user
.
> From: rmetz...@apache.org
> Date: Mon, 27 Jul 2015 14:01:11 +0200
> Subject: Re: add some new api to the scheduler in the job manager
> To: dev@flink.apache.org
>
> Hi MaGuoWei,
>
> would you like to have done automatically by Flink or based on some user
> inpu
..@apache.org
> > > Date: Mon, 27 Jul 2015 05:14:00 +
> > > Subject: Re: add some new api to the scheduler in the job manager
> > > To: dev@flink.apache.org
> > >
> > > Hi,
> > > I think for more details on giving back resources of a running cl
uoWei wrote:
> It is great! Is there any document? I am very interested in this.
> thanks
>
>
> > From: aljos...@apache.org
> > Date: Mon, 27 Jul 2015 05:14:00 +0000
> > Subject: Re: add some new api to the scheduler in the job manager
> > To: dev@flink.apache.org
It is great! Is there any document? I am very interested in this.
thanks
> From: aljos...@apache.org
> Date: Mon, 27 Jul 2015 05:14:00 +
> Subject: Re: add some new api to the scheduler in the job manager
> To: dev@flink.apache.org
>
> Hi,
> I think for more details on
Hi,
I think for more details on giving back resources of a running cluster we
have to wait for Robert's opinion. In the mean time, you can also just run
a single job that will bring up some yarn containers and then release them
afterward using this:
https://ci.apache.org/projects/flink/flink-docs-r
12 matches
Mail list logo
