Uma Maheswara Rao G created HDFS-15354:
------------------------------------------
Summary: clearCorruptLazyPersistFiles incrementalBlock removal
should be out side write lock
Key: HDFS-15354
URL: https://issues.apache.org/jira/browse/HDFS-15354
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 3.2.1
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
In LazyPersistFileScrubber#clearCorruptLazyPersistFiles collecting blocks for
removal and also removing them in write lock.
removeBlocks should be moved out of writelock as removeBlocks has incremental
deletion logic in which it will acquire write lock and unlock for every block
removal.
If there are more corrupt blocks to remove in cluster, it may hold write lock
for longer time.
{code:java}
for (BlockCollection bc : filesToDelete) {
LOG.warn("Removing lazyPersist file " + bc.getName() + " with no
replicas.");
BlocksMapUpdateInfo toRemoveBlocks =
FSDirDeleteOp.deleteInternal(
FSNamesystem.this,
INodesInPath.fromINode((INodeFile) bc), false);
changed |= toRemoveBlocks != null;
if (toRemoveBlocks != null) {
removeBlocks(toRemoveBlocks); // Incremental deletion of blocks
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
