Sergey Soldatov created HDDS-17651:
--------------------------------------
Summary: Intermittent failure in
TestQueryNode#testStaleNodesCount: datanode stop exceeds the 10 s wait
Key: HDDS-17651
URL: https://issues.apache.org/jira/browse/HDDS-17651
Project: Apache Ozone
Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
h3. Problem
{{TestQueryNode#testStaleNodesCount}} fails intermittently:
{noformat}
java.util.concurrent.TimeoutException: Timed out waiting for condition.
at
org.apache.hadoop.hdds.scm.node.TestQueryNode.testStaleNodesCount(TestQueryNode.java:90)
{noformat}
Example:
https://github.com/apache/ozone/actions/runs/41325666401/job/124908346043
The test stops two datanodes one after the other, then gives SCM 10 s to report
both as STALE or DEAD. A datanode stop normally takes about 2 s, so 10 s is
usually enough.
h3. Cause
A datanode can become a Ratis pipeline leader just as it is being stopped.
Ratis then waits up to 3 s for each connection to its followers to close. In
the failed run this made the first stop take 8 s instead of 2 s, and the 10 s
wait ran out.
This is a timing problem in the test, not a regression: the Ratis shutdown
behavior has not changed. RATIS-2784 tracks making these timeouts configurable.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
