[
https://issues.apache.org/jira/browse/KUDU-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18147525#comment-18147525
]
ASF subversion and git services commented on KUDU-3830:
-------------------------------------------------------
Commit 15c4979b8c08b928e35f695f4501d71fd925619d in kudu's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=15c4979b8 ]
KUDU-3830: Add RWFileOptions::random_access to disable readahead
If set, PosixEnv::NewRWFile() advises the kernel with
posix_fadvise(POSIX_FADV_RANDOM) that the file is read at random
offsets, so the kernel doesn't read ahead when reading from it. The
advice applies only to the newly opened file description; writes and
hole punching are not affected. A failure of posix_fadvise() is logged
as a warning since it affects only performance, not correctness.
On platforms other than Linux, the option is ignored and a warning is
logged once.
The default is false, so the behavior of existing callers doesn't
change.
The motivation comes from Impala's remote data cache, which uses
kudu::RWFile for its backing files and reads mainly random offsets.
On tpcds 10 TB benchmark with NVMe drive + RHEL 8.6 this reduced
IO wait time ~40%.
Change-Id: I394ef038c5dc21752fbadf9eda48dc933cc56aa9
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Reviewed-on: https://gerrit.cloudera.org/25464
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Ashwani Raina <[email protected]>
Tested-by: Ashwani Raina <[email protected]>
> Allow fadvise POSIX_FADV_RANDOM on RWFile to avoid readahead
> ------------------------------------------------------------
>
> Key: KUDU-3830
> URL: https://issues.apache.org/jira/browse/KUDU-3830
> Project: Kudu
> Issue Type: Improvement
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> Impala uses RWFile as remote data cache (e.g. cache S3 reads to nvme).
> Turning off readahead almost double IO speed on RHEL 8 +m6id.4xl.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)