Lin Xiao created HDFS-5389: ------------------------------ Summary: Remove INode limitations in Namenode Key: HDFS-5389 URL: https://issues.apache.org/jira/browse/HDFS-5389 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 0.23.1 Reporter: Lin Xiao Priority: Minor
Current HDFS Namenode stores all of its metadata in RAM. This has allowed Hadoop clusters to scale to 100K concurrent tasks. However, the memory limits the total number of files that a single NN can store. While Federation allows one to create multiple volumes with additional Namenodes, there is a need to scale a single namespace and also to store multiple namespaces in a single Namenode. When inodes are also stored on persistent storage, the system's boot time can be significantly reduced because there is no need to replay edit logs. It also provides the potential to support extended attributes once the memory size is not the bottleneck. -- This message was sent by Atlassian JIRA (v6.1#6144)