[
https://issues.apache.org/jira/browse/HADOOP-19596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18012497#comment-18012497
]
Anuj Modi commented on HADOOP-19596:
------------------------------------
Thanks for the feedback. This is definitely something we should consider.
As a part of this change I will only target at improving the performance of
Prefetches. When and how much to prefetch we won't change until we consider the
user suggested read policy.
We will take that improvement as a separate work item and plan that dilgently
for all read policies including sequential.
Here is the new Jira to consider user suggested read policy and adjust our read
related optimizations
accordingly.https://issues.apache.org/jira/browse/HADOOP-19647
Holding on the Jira proposing changes in Input Stream for now and will take up
with above only: https://issues.apache.org/jira/browse/HADOOP-19641
> ABFS: [ReadAheadV2] Increase Prefetch Aggressiveness to improve sequential
> read performance
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-19596
> URL: https://issues.apache.org/jira/browse/HADOOP-19596
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.5.0, 3.4.1
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Attachments: Read Buffer Manager V2.pdf
>
>
> Various analyses done in the past have shown a need for significant
> improvement in the performance of sequential reads. The current
> implementation clearly shows the lack of parallelism that is needed to cater
> to high throughput sequential read workloads.
> More details on updated design and results of POC benchmarking will be added
> here soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
