[
https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861863#comment-16861863
]
Yingjie Cao commented on FLINK-12070:
-------------------------------------
[~StephanEwen] The total memory of network buffer pool per taskmanager is 5G,
for the test case incurring the biggest performance regression (it cost more
than 4 times of time for the map stage), the total size of output data per
taskmanager is less than 5G (more than 4G), because the buffer pool of blocking
result partition is unbounded, all network memory can be used to cache the
output data. I confirmed that by checking the IO speed of the test machine, and
there is no disk IO when map task is running.
When the data volume is small (less than 1G per taskmanager) or the data volume
is large (more than 5G per taskmanager). I guess that before the data volume
reach 5G per taskmanager, the larger the data volume the more performance
regression and when the data volume is small enough, the total time used is
also small and other overhead (including performance jitter) dominates the
cost, so there is no visible regression.
By the way, the mmap writing is also much slower for these cases.
> Make blocking result partitions consumable multiple times
> ---------------------------------------------------------
>
> Key: FLINK-12070
> URL: https://issues.apache.org/jira/browse/FLINK-12070
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Affects Versions: 1.9.0
> Reporter: Till Rohrmann
> Assignee: Stephan Ewen
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Attachments: image-2019-04-18-17-38-24-949.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In order to avoid writing produced results multiple times for multiple
> consumers and in order to speed up batch recoveries, we should make the
> blocking result partitions to be consumable multiple times. At the moment a
> blocking result partition will be released once the consumers has processed
> all data. Instead the result partition should be released once the next
> blocking result has been produced and all consumers of a blocking result
> partition have terminated. Moreover, blocking results should not hold on slot
> resources like network buffers or memory as it is currently the case with
> {{SpillableSubpartitions}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
