sumitagrawl commented on code in PR #8788:
URL: https://github.com/apache/ozone/pull/8788#discussion_r2215965854
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ContainerEndpoint.java:
##########
@@ -583,29 +583,28 @@ public Response getContainerMisMatchInsights(
new ArrayList<>();
Long minContainerID = prevKey + 1;
Iterator<ContainerInfo> scmNonDeletedContainers =
- containerManager.getContainers().stream()
- .filter(containerInfo -> (containerInfo.getContainerID()
>= minContainerID))
- .filter(containerInfo -> containerInfo.getState() !=
HddsProtos.LifeCycleState.DELETED)
-
.sorted(Comparator.comparingLong(ContainerInfo::getContainerID)).iterator();
- ContainerInfo scmContainerInfo = scmNonDeletedContainers.hasNext() ?
- scmNonDeletedContainers.next() : null;
+
containerManager.getContainerInfoIterator(ContainerID.valueOf(minContainerID),
+ containerInfo -> containerInfo.getState() !=
HddsProtos.LifeCycleState.DELETED);
+ ContainerInfo scmContainerInfo = scmNonDeletedContainers.hasNext() ?
scmNonDeletedContainers.next() : null;
DataFilter dataFilter = DataFilter.fromValue(missingIn.toUpperCase());
try (SeekableIterator<Long, ContainerMetadata> omContainers =
- reconContainerMetadataManager.getContainersIterator()) {
+ reconContainerMetadataManager.getContainersIterator()) {
omContainers.seek(minContainerID);
- ContainerMetadata containerMetadata = omContainers.hasNext() ?
omContainers.next() : null;
+
switch (dataFilter) {
case SCM:
List<ContainerMetadata> notSCMContainers = new ArrayList<>();
- while (containerMetadata != null && notSCMContainers.size() < limit) {
- Long omContainerID = containerMetadata.getContainerID();
+ while (omContainers.hasNext() && notSCMContainers.size() < limit) {
Review Comment:
Current code logic is not efficient, can discard existing logic, and have
mechanism as below,
Pre-condition: SCM container Info is in memory, OM container can be iterated
from containerKeyCountTable.
SCM: (missing in SCM, data loss):
- init with (null if no last key OR last key)
- Iterate over OM containers (can use keyIterator to deserialize only key)
- check in SCM container Map, if exist and not deleted filter
- If nonSCMContainer size reaches limit, break and return
Performance improvement:
1. SCM map sorting are not required
Limitation:
- it will return all containers in lexographical order (not necessary in
numeric order). But this ordering is not the target, so no issue.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ContainerEndpoint.java:
##########
@@ -627,39 +626,35 @@ public Response getContainerMisMatchInsights(
case OM:
List<ContainerInfo> nonOMContainers = new ArrayList<>();
while (scmContainerInfo != null && nonOMContainers.size() < limit) {
- Long omContainerID = containerMetadata == null ? null :
containerMetadata.getContainerID();
+ Long omContainerID = omContainers.peekNextKey();
Review Comment:
OM case,
- sort scm removing deleted containers.
iterate scm,
- check via seek() as current OR via isExist(), and identify which is
better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
