Zheng Hu created HDFS-14541:
-------------------------------
Summary: ShortCircuitReplica#unref cost about 6% cpu and 6% heap
allocation because of the frequent thrown NoSuchElementException in our HBase
benchmark
Key: HDFS-14541
URL: https://issues.apache.org/jira/browse/HDFS-14541
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Zheng Hu
Attachments: async-prof-pid-94152-alloc-2.svg,
async-prof-pid-94152-cpu-1.svg
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and found that there're some performence issues in DFSClient
now .
See the attached two flame graph, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
should remove this from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
