Siyao Meng created HDDS-17123:
---------------------------------
Summary: Add integration tests for 5-node OM HA and SCM HA
Key: HDDS-17123
URL: https://issues.apache.org/jira/browse/HDDS-17123
Project: Apache Ozone
Issue Type: Sub-task
Components: OM HA, test, SCM HA
Reporter: Siyao Meng
Add MiniOzoneCluster integration coverage for 5-node OM and SCM rings:
* *TestOzoneManagerHAWithFiveNodes* (new): 5 OMs form one Raft group; writes
reach all 5 OM DBs; leader transfer to a specific OM; stop the leader and a
follower -> new leader, reads and writes work; stop a 3rd OM -> no leader for 3
election timeouts and writes fail; restart one -> quorum restored; restart the
last 2 after the leader purged its log -> both install a checkpoint and
converge; the ring survives another leader failure.
* *TestStorageContainerManagerHAWithFiveNodes* (new): the same for 5 SCMs, plus
every datanode has 5 SCM endpoints and every SCM sees all datanodes HEALTHY;
container allocations and key writes reach all 5 SCMs.
* *TestAddRemoveOzoneManager#testBootstrapToFiveOMsAndDecommissionToThree*
(new): bootstrap 3 -> 5 OMs, stop 2 original OMs so the majority needs the
votes of both bootstrapped OMs, decommission back to 3.
* *TestSecretKeysApi#testSecretKeyAfterSCMFailover*: run with 3 and 5 secure
SCMs, stop the leader (n-1)/2 times, and check every SCM certificate is issued
by the shared root CA.
Also fix {{MiniOzoneHAClusterImpl#stopSCM}}: it called
{{stopAndDeactivate(StorageContainerManager)}}, which called {{stopSCM}} again,
so it always failed with StackOverflowError. Nothing used it before.
{{TestStorageContainerManagerHAWithFiveNodes#testStoppedSCMsCatchUpViaCheckpointInstall}}
needs HDDS-16924: without it the test fails with ~12,000 "Rocks Database is
closed" errors; with it the test passes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
