[
https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17836108#comment-17836108
]
ASF GitHub Bot commented on HADOOP-19096:
-----------------------------------------
anujmodi2021 opened a new pull request, #6720:
URL: https://github.com/apache/hadoop/pull/6720
Description of PR
Jira: https://issues.apache.org/jira/browse/HADOOP-19096
Commit in trunk:
https://github.com/apache/hadoop/commit/dbe2d612586d7b78d61ef64c706eccf7fbf6f35c
ABFS has a client-side throttling mechanism which works on the metrics
collected from past requests made. I requests are getting failed due to
throttling at server, we update our metrics and client side backoff is
calculated based on those metrics.
This PR enhances the logic to decide which requests should be considered to
compute client side backoff interval as follows:
For each request made by ABFS driver, we will determine if they should
contribute to Client-Side Throttling based on the status code and result:
Status code in 2xx range: Successful Operations should contribute.
Status code in 3xx range: Redirection Operations should not contribute.
Status code in 4xx range: User Errors should not contribute.
Status code is 503: Throttling Error should contribute only if they are due
to client limits breach as follows:
503, Ingress Over Account Limit: Should Contribute
503, Egress Over Account Limit: Should Contribute
503, TPS Over Account Limit: Should Contribute
503, Other Server Throttling: Should not Contribute.
Status code in 5xx range other than 503: Should not Contribute.
IOException and UnknownHostExceptions: Should not Contribute.
> [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic
> --------------------------------------------------------------
>
> Key: HADOOP-19096
> URL: https://issues.apache.org/jira/browse/HADOOP-19096
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.1
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.1
>
>
> ABFS has a client-side throttling mechanism which works on the metrics
> collected from past requests made. I requests are getting failed due to
> throttling at server, we update our metrics and client side backoff is
> calculated based on those metrics.
> This PR enhances the logic to decide which requests should be considered to
> compute client side backoff interval as follows:
> For each request made by ABFS driver, we will determine if they should
> contribute to Client-Side Throttling based on the status code and result:
> # Status code in 2xx range: Successful Operations should contribute.
> # Status code in 3xx range: Redirection Operations should not contribute.
> # Status code in 4xx range: User Errors should not contribute.
> # Status code is 503: Throttling Error should contribute only if they are
> due to client limits breach as follows:
> ## 503, Ingress Over Account Limit: Should Contribute
> ## 503, Egress Over Account Limit: Should Contribute
> ## 503, TPS Over Account Limit: Should Contribute
> ## 503, Other Server Throttling: Should not Contribute.
> # Status code in 5xx range other than 503: Should not Contribute.
> # IOException and UnknownHostExceptions: Should not Contribute.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
