Dear Ufuk,
the wiki entry is exactly what i was looking for. I found it quite
complicated to understand on a first attempt but i will dedicate some more
time for it in the future.
Thanks.
Regards
Leon
1. Jun 2016 13:06 by u...@apache.org:
> There is this in the Wiki:
> https://cwiki.apache.
There is this in the Wiki:
https://cwiki.apache.org/confluence/display/FLINK/Data+exchange+between+tasks
Buffers for data exchange come from the network buffer pool (by
default 2048 * 32KB buffers). They are distributed to the running
tasks and each logical channel between tasks needs at least one
I have a question regarding how tuples are buffered between (possibly
chained) subtasks.
Is it correct that there is a buffer for each vertex in the DAG of subtasks?
Regardless of task slot sharing? If yes, then the primary optimization in
this regard is operator chaining.
Furthermore, how do
> On 04 Feb 2016, at 12:02, Gwenhael Pasquiers
> wrote:
>
> Ok thanks !
>
> All that’s left is to wait then.
If you have spare time and are working with the current snapshot version, it
would be great to get some feedback on the pull request. :-)
– Ufuk
Ok thanks !
All that’s left is to wait then.
B.R.
From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of Stephan
Ewen
Sent: jeudi 4 février 2016 11:19
To: user@flink.apache.org
Subject: Re: Internal buffers supervision and yarn vCPUs
Concerning the first question:
What you
Concerning the first question:
What you are looking for is backpressure monitoring. If a task cannot push
its data to the next task, it is backpressured.
This pull request adds a first version of backpressure monitoring:
https://github.com/apache/flink/pull/1578
We will try and get it merged soo
Hi Gwen,
let me answer the second question: There is a JIRA to reintroduce the
configuration parameter: https://issues.apache.org/jira/browse/FLINK-2213.
I will try to get a fix for this into the 1.0 release.
I think I removed back then because users were unable to define the number
of vcores ind
Hi,
I’ve got two more questions on different topic…
First one :
Is there a way to monitor the buffers status. In order to find bottleneck in
our application we though it could be usefull to be able to have a look at the
different exchange buffers’ status. To know if they are full (or as an exa
