Joe McDonnell created IMPALA-14445:
--------------------------------------
Summary: Improve error message for RPC backpressure and other KRPC
error messages
Key: IMPALA-14445
URL: https://issues.apache.org/jira/browse/IMPALA-14445
Project: IMPALA
Issue Type: Task
Components: Distributed Exec
Affects Versions: Impala 5.0.0
Reporter: Joe McDonnell
On a cluster, we saw queries failing with errors like this:
{noformat}
Exec() rpc failed: Remote error: Service unavailable: ExecQueryFInstances
request on impala.ControlService from 1.2.3.4:12345 dropped due to
backpressure. The service queue contains 0 items out of a maximum of
2147483647; memory consumption is 0.{noformat}
In this case, the executor is rejecting a message from the coordinator
(1.2.3.4:12345 is the IP:port of a coordinator) and the query is failing due to
that. The error message doesn't make it clear that the executor is the one
rejecting the message, so this can lead to focusing on the coordinator rather
than tracking down the executor. Also, the message should include the executor
hostname to make it easy to know where to look.
The executor black-listing / query retry logic constructs a better message for
the retryable status, but we don't use that for backpressure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
