[jira] Resolved: (HDFS-1452) ant compile-contrib is broken

[ https://issues.apache.org/jira/browse/HDFS-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik resolved HDFS-1452. -- Resolution: Fixed I have checked that contrib compilation is back to normal and isn't af

Re: Reason to store 64 block file in a sub directory?

Thanks guys, That makes more sense to me now. On Mon, Oct 11, 2010 at 4:59 PM, Dhruba Borthakur wrote: > The number is just an adhoc number. The policy is not to put too many block > files in the same directory because some local filesystems behave badly if > the number of files in the same dir

[jira] Created: (HDFS-1452) ant compile-contrib is broken

ant compile-contrib is broken - Key: HDFS-1452 URL: https://issues.apache.org/jira/browse/HDFS-1452 Project: Hadoop HDFS Issue Type: Bug Components: contrib/hdfsproxy Affects Versions: 0.22.0

Re: Reason to store 64 block file in a sub directory?

The number is just an adhoc number. The policy is not to put too many block files in the same directory because some local filesystems behave badly if the number of files in the same directory exceed a certain value. -dhruba On Mon, Oct 11, 2010 at 1:15 PM, Thanh Do wrote: > Hi all, > > can an

Re: Reason to store 64 block file in a sub directory?

If I recall correctly, ext3 has O(n) performance for lookup of a directory entry. So, having thousands of files in a directory is bad for performance. Additionally, there's a max of 31998 files in a directory, so you have to split into subdirs eventually. -Todd On Mon, Oct 11, 2010 at 8:15 PM, Th

Reason to store 64 block file in a sub directory?

Hi all, can anyone explain to me while do HDFS has the policy to store 64 block files in a single sub directory? and if the number of block files increase, it just simply creates another subdir and put the block files there. Thanks Thanh

[jira] Created: (HDFS-1451) Extend the pluggable block placement policy interface to handle/allow file-level intelligence

Extend the pluggable block placement policy interface to handle/allow file-level intelligence - Key: HDFS-1451 URL: https://issues.apache.org/jira/browse/HDFS-1451

[jira] Created: (HDFS-1450) HDFS federation: Introduce block pool ID into FSDatasetInterface

HDFS federation: Introduce block pool ID into FSDatasetInterface Key: HDFS-1450 URL: https://issues.apache.org/jira/browse/HDFS-1450 Project: Hadoop HDFS Issue Type: New Featur

[jira] Created: (HDFS-1449) HDFS federation: Fix unit test failures

HDFS federation: Fix unit test failures --- Key: HDFS-1449 URL: https://issues.apache.org/jira/browse/HDFS-1449 Project: Hadoop HDFS Issue Type: Bug Affects Versions: Federation Branch Repo