[
https://issues.apache.org/jira/browse/FLINK-28373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingjie Cao resolved FLINK-28373.
---------------------------------
Resolution: Fixed
Merged into master via d6a47d897a9a4753c800b39adb17c06e154422cc
> Read a full buffer of data per file IO read request for sort-shuffle
> --------------------------------------------------------------------
>
> Key: FLINK-28373
> URL: https://issues.apache.org/jira/browse/FLINK-28373
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Yingjie Cao
> Assignee: Yuxin Tan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.16.0
>
>
> Currently, for sort blocking shuffle, the corresponding data readers read
> shuffle data in buffer granularity. Before compression, each buffer is 32K by
> default, after compression the size will become smaller (may less than 10K).
> For file IO, this is pretty smaller. To achieve better performance and reduce
> IOPS, we can read more data per IO read request and parse buffer header and
> data in memory.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
