pnowojski commented on a change in pull request #11564: [FLINK-16864][metrics]
Add IdleTime metric for task.
URL: https://github.com/apache/flink/pull/11564#discussion_r400801364
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java
##########
@@ -276,6 +281,21 @@ protected void checkErroneous() throws IOException {
}
}
+ protected void addBufferConsumer(BufferConsumer consumer, int
targetChannel) throws IOException {
+ targetPartition.addBufferConsumer(consumer, targetChannel);
+ }
+
+ protected BufferBuilder getBufferBuilder() throws IOException,
InterruptedException {
+ BufferBuilder builder;
+ if (!targetPartition.isApproximatelyAvailable()) {
+ long start = System.currentTimeMillis();
+ builder = targetPartition.getBufferBuilder();
+
idleTimeMsPerSecond.markEvent(System.currentTimeMillis() - start);
+ } else {
+ builder = targetPartition.getBufferBuilder();
+ }
+ return builder;
+ }
Review comment:
I think this is most of the times correct, however there might be a race
condition with `NetworkBufferPool#redistributeBuffers` and
`isApproximatelyAvailable()` check. Check might return available, while the
method would still block.
The proper way of doing it would be to add non blocking version of the
method `getBufferBuilder()` (or rather rename the existing one to
`getBufferBuilderBlocking()` and implement new `getBufferBuilder()` which would
be non blocking) to the `ResultPartition` (and `BufferPool` +
`LocalBufferPool`). It should be easy, as there are already non blocking
methods for requesting memory segments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
