[
https://issues.apache.org/jira/browse/IMPALA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu updated IMPALA-14466:
-----------------------------
Description:
The {{RemoteAdmissionControlClient}} gets the service proxy for {{admissiond}}
once before entering the retry loop. If the {{admissiond}} service restarts and
its endpoint IP changes, the client will endlessly retry against the old, stale
IP address, causing the query to fail with a timeout.
Reproduce{*}:{*}
# Start with {{admissiond}} service
# Change the admission_service_host, let's say it is set to "localhost", we
change the localhost to inaccessible ip 127.0.0.2 in /etc/hosts.
# Submit a query, and it will be blocked in the retry logic.
# Change the localhost back to 127.0.0.1 in /etc/hosts.
# *Expected:* The client's retry logic should discover the new {{admissiond}}
ip and succeed.
# *Actual:* The client continues retrying the old IP and the query times out.
Move the {{AdmissionControlService::GetProxy()}} call inside the {{while}} loop
could be a solution.
was:
The {{RemoteAdmissionControlClient}} gets the service proxy for {{admissiond}}
*once* before entering the retry loop. If the {{admissiond}} service restarts
and its endpoint IP changes, the client will endlessly retry against the old,
stale IP address, causing the query to fail with a timeout.
Reproduce{*}:{*}
# Start with {{admissiond}} service
# Change the admission_service_host, let's say it is set to "localhost", we
change the localhost to inaccessible ip 127.0.0.2 in /etc/hosts.
# Submit a query, and it will be blocked in the retry logic.
# Change the localhost back to 127.0.0.1 in /etc/hosts.
# *Expected:* The client's retry logic should discover the new {{admissiond}}
ip and succeed.
# *Actual:* The client continues retrying the old IP and the query times out.
Move the {{AdmissionControlService::GetProxy()}} call inside the {{while}} loop
could be a solution.
> Admissiond remote client retry may not work if ip changes
> ---------------------------------------------------------
>
> Key: IMPALA-14466
> URL: https://issues.apache.org/jira/browse/IMPALA-14466
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Yida Wu
> Priority: Major
>
> The {{RemoteAdmissionControlClient}} gets the service proxy for
> {{admissiond}} once before entering the retry loop. If the {{admissiond}}
> service restarts and its endpoint IP changes, the client will endlessly retry
> against the old, stale IP address, causing the query to fail with a timeout.
> Reproduce{*}:{*}
> # Start with {{admissiond}} service
> # Change the admission_service_host, let's say it is set to "localhost", we
> change the localhost to inaccessible ip 127.0.0.2 in /etc/hosts.
> # Submit a query, and it will be blocked in the retry logic.
> # Change the localhost back to 127.0.0.1 in /etc/hosts.
> # *Expected:* The client's retry logic should discover the new
> {{admissiond}} ip and succeed.
> # *Actual:* The client continues retrying the old IP and the query times out.
> Move the {{AdmissionControlService::GetProxy()}} call inside the {{while}}
> loop could be a solution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
