xichen01 commented on code in PR #9696:
URL: https://github.com/apache/ozone/pull/9696#discussion_r2879602667
##########
hadoop-ozone/freon/src/main/java/org/apache/hadoop/ozone/freon/OmMetadataGenerator.java:
##########
@@ -165,18 +183,42 @@ public Void call() throws Exception {
OzoneConfiguration conf = createOzoneConfiguration();
replicationConfig = replication.fromParamsOrConfig(conf);
- try (OzoneClient rpcClient = createOzoneClient(omServiceID, conf)) {
- ensureVolumeAndBucketExist(rpcClient, volumeName, bucketName);
- ozoneManagerClient = createOmClient(conf, omServiceID);
- bucket = rpcClient.getObjectStore().getVolume(volumeName)
- .getBucket(bucketName);
+ if (clientsNo <= 0) {
+ clientsNo = 1;
+ }
+
+ try (OzoneClient ozClient = createOzoneClient(omServiceID, conf)) {
+ ensureVolumeAndBucketExist(ozClient, volumeName, bucketName);
+
+ ozoneClients = new OzoneClient[clientsNo];
+ List<String> followerOMNodeIds = null;
+ int currentFollowerAffinityIndex = 0;
+ if (enableFollowerAffinity) {
+ followerOMNodeIds = getOMFollowerNodeIds(ozClient);
+ }
+ for (int i = 0; i < clientsNo; i++) {
+ OzoneClient ozoneClient = createOzoneClient(omServiceID, conf);
+ if (enableFollowerAffinity) {
+ // Point the client proxy to the followers in a round-robin fashion
+ // This balances the read loads on the OM followers
+ changeInitialProxyForFollowerRead(ozoneClient,
+ followerOMNodeIds.get(currentFollowerAffinityIndex));
+ currentFollowerAffinityIndex = (currentFollowerAffinityIndex + 1) %
followerOMNodeIds.size();
+ }
+ ozoneClients[i] = ozoneClient;
Review Comment:
Perhaps we could add a log to print the OM connection for each client
connection (such as: client1 -> OM1, client2- OM2).
This way, when testing Follower read operations, we can verify whether all
followers have successfully connected.
##########
hadoop-ozone/freon/src/main/java/org/apache/hadoop/ozone/freon/OmMetadataGenerator.java:
##########
@@ -464,4 +522,44 @@ enum Operation {
INFO_VOLUME,
MIXED,
}
+
+ private List<String> getOMFollowerNodeIds(OzoneClient ozoneClient) throws
IOException {
+ List<OMRoleInfo> omRoleInfos = ozoneClient.getProxy().getOmRoleInfos();
+ String leaderOMNodeId = null;
+ List<String> followerOMNodeIds = new ArrayList<>();
+ for (OMRoleInfo omRoleInfo : omRoleInfos) {
+ if (omRoleInfo.getServerRole().equals("LEADER")) {
+ if (leaderOMNodeId != null) {
+ // This situation should be rare
+ throw new IllegalStateException("There are more than one leader
detected, please retry again");
+ }
+ leaderOMNodeId = omRoleInfo.getNodeId();
+ } else if (omRoleInfo.getServerRole().equals("FOLLOWER")) {
+ followerOMNodeIds.add(omRoleInfo.getNodeId());
+ }
+ }
+ if (followerOMNodeIds.isEmpty()) {
+ throw new IllegalArgumentException("There is no follower in the OM
service, please retry again");
+ }
+ return followerOMNodeIds;
+ }
+
+ private void changeInitialProxyForFollowerRead(OzoneClient ozoneClient,
String omNodeId) {
+ OzoneManagerProtocolClientSideTranslatorPB ozoneManagerClient =
+ (OzoneManagerProtocolClientSideTranslatorPB)
+ ozoneClient.getProxy().getOzoneManagerClient();
+
+ OmTransport transport = ozoneManagerClient.getTransport();
+
+ if (transport instanceof Hadoop3OmTransport) {
Review Comment:
Can throw a exception in the else branch, the current failure is silent
##########
hadoop-ozone/freon/src/main/java/org/apache/hadoop/ozone/freon/OmMetadataGenerator.java:
##########
@@ -311,14 +353,21 @@ private void applyOperation(long counter) throws
Exception {
counter = ThreadLocalRandom.current().nextLong(getTestNo());
}
final String keyName = getPath(counter);
+ // Use counter instead of thread sequence ID so that a single operation
can be
+ // executed by different clients in mixed operations scenario
+ final OzoneClient ozoneClient = getOzoneClient(counter);
+ final ClientProtocol clientProtocol = ozoneClient.getProxy();
+ final OzoneManagerProtocol ozoneManagerClient =
clientProtocol.getOzoneManagerClient();
Review Comment:
In the future, it may be possible to separately record performance metrics
for different OMs. This allows for better observation of the performance
differences between different OMs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
