pnowojski commented on a change in pull request #11877:
URL: https://github.com/apache/flink/pull/11877#discussion_r665900515
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java
##########
@@ -164,6 +168,27 @@ void addCreditOrResumeConsumption(
}
}
+ /**
+ * Announces remaining backlog to the consumer after the available data
notification or data
+ * consumption resumption.
+ */
+ private void announceBacklog(NetworkSequenceViewReader reader) {
+ int backlog = reader.getRemainingBacklog();
+ if (backlog > 0) {
Review comment:
`NetworkSequenceViewReader#isAvailable()` has the same problem of
synchronisation costs.
> I think there is one thing blocking us from reusing the backlog from
PartitionRequestQueue#notifyReaderNonEmpty(reader): we have duplicated
availability notifications. We may get outdated backlog.
What is the problem? Keep in mind that spurious notifications shouldn't be
that big of a problem. If we sometimes wake up too many times, and we rarely
send incorrect backlog information that should be fine, as long as we always
over estimate the backlog.
After all even in your version I think there can be a race condition where
`PartitionRequestQueue` is notified reader is non empty, you check the
`getRemainingBacklog()` and send the `BacklogAnnouncement` message, while
before it gets processed by the receiver, some buffer is polled from this
reader and backlog goes down to `0`. And as a result, receiver assigns as a
credit to a sender that doesn't have any data anymore?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
