Great, thanks for letting me know. pt., 19 mar 2021 o 20:24 Alexey Trenikhun <[email protected]> napisał(a):
> Hi Piotrek, > > Thank for information, looks like isBackPressured ( and in > future backPressuredTimeMsPerSecond) is more useful for our simple > monitoring purposes. Looking forward for updated blog post > > Thanks, > Alexey > > ------------------------------ > *From:* Piotr Nowojski <[email protected]> > *Sent:* Friday, March 19, 2021 5:01 AM > *To:* Alexey Trenikhun <[email protected]> > *Cc:* Flink User Mail List <[email protected]> > *Subject:* Re: inputFloatingBuffersUsage=0? > > Hi, > > clarification about the 2nd part. Required memory is one single exclusive > buffer per channel, so if you are running low on memory, floating buffers > are one of the first to go, hence you could observe some `0` for floating > buffers. > > For analysing backpressure I would recommend to use a different metric: > `isBackPressured`. It was added in Flink 1.10, after this blog post had > been written. Since 1.11 we also have `idleTimeMsPerSecond` that might be > helpful. > > In 1.13 all of this will become even more simple, with changes to Web UI > and an addition of `backPressuredTimeMsPerSecond` and > `busyTimeMsPerSecond`. We are planning to release a new updated blog post > about analysing backpressure in the following weeks. > > Best, > Piotrek > > pt., 19 mar 2021 o 11:57 Piotr Nowojski <[email protected]> napisał(a): > > Hi Alexey, > > Have you looked at the documentation [1]? > > > inPoolUsage An estimate of the input buffers usage. (ignores > LocalInputChannels) Gauge > > inputFloatingBuffersUsage An estimate of the floating input buffers > usage. (ignores LocalInputChannels) Gauge > > inputExclusiveBuffersUsage An estimate of the exclusive input buffers > usage. (ignores LocalInputChannels) Gauge > > Second reason might be simply that your job/task didn't manage to > acquire all of the recommended memory. Recommended memory is > `floatingBuffers + exclusiveBuffers * numberOfChannels`. Required memory is > just `1 * numberOfChannels`. If there is not enough memory to satisfy > recommended values, Flink will work with anything between recommended and > required amounts. > > Best, > Piotrek > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#default-shuffle-service > > czw., 18 mar 2021 o 08:30 Alexey Trenikhun <[email protected]> napisał(a): > > Hello, > While trying to investigate back pressure using hints from [1], I've > noticed that flink_taskmanager_job_task_buffers_inPoolUsage > and flink_taskmanager_job_task_buffers_inputFloatingBuffersUsage are always > 0, which looks suspicious, are these metrics still populated ? > > Thanks, > Alexey > > [1] https://flink.apache.org/2019/07/23/flink-network-stack-2.html > Apache Flink: Flink Network Stack Vol. 2: Monitoring, Metrics, and that > Backpressure Thing > <https://flink.apache.org/2019/07/23/flink-network-stack-2.html> > Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure > Thing. 23 Jul 2019 Nico Kruber & Piotr Nowojski . In a previous blog post, > we presented how Flink’s network stack works from the high-level > abstractions to the low-level details.This second blog post in the series > of network stack posts extends on this knowledge and discusses monitoring > network-related metrics to identify ... > flink.apache.org > >
