Anil,
I actually moved your mail into hdfs-dev@ earlier and also responded.
But anyway, re-posting since you perhaps did not notice it:
The transfer thread load is definitely considered when building the DN
replication pipeline. See the method
ReplicationTargetChooser#isGoodTarget(…), which is ca
Scenario: If I run huge number of jobs(all these jobs will use the same
resources(input files)) on mini cluster(say 10-15 nodes), then every time
namenode returning the first block of nearest data node. So in this case all
the clients are trying to do read/write operations on same block.
So is the
Anil,
Moving this to hdfs-dev@.
The transfer thread load is definitely considered when building the DN
replication pipeline. See the method
ReplicationTargetChooser#isGoodTarget(…), which is called during each
type of choice (local node, local rack, remote rack or random). One
part of its analysi
