Sumit Agrawal created HDDS-13639:
------------------------------------
Summary: optimize container iterator for frequent operation
Key: HDDS-13639
URL: https://issues.apache.org/jira/browse/HDDS-13639
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Sumit Agrawal
One one of environment having 900K containers in a datanode, looping to the
list takes {*}2.5 minutes{*}, which is very slow.
Need check for below cases:
1. Can avoid this as used for only metrics, or better optimize the code for
loop for each volume
ContainerController.getContainerCount(HddsVolume)
(org.apache.hadoop.ozone.container.ozoneimpl)
HddsVolume.getContainers() (org.apache.hadoop.ozone.container.common.volume)
VolumeInfoMetrics.getContainers()
(org.apache.hadoop.ozone.container.common.volume)
2. can avoid copy of containerSet, and use direct map iterator
org.apache.hadoop.ozone.container.common.impl.ContainerSet#getContainerReport
This is frequest of metric polled by prometheus is generally 30sec.
{code:java}
List<Container<?>> containers = new ArrayList<>(containerMap.values()); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
