Block placement Policy

2013-05-04 Thread Mohammad Mustaqeem 
I have read somewhere that a user can specified his own ReplicaPlacementPolicy.
How can I specify my own ReplicaPlacementPolicy?
I you have any sample ReplicaPlacementPolicy, then please share it..

-- 
*With regards ---*
*Mohammad Mustaqeem*,
M.Tech (CSE)
MNNIT Allahabad
9026604270

Re: Block placement Policy

2013-05-04 Thread Rajat Sharma 
Have a look at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.
This is default placement policy, you should be able to extend this to
implement your own policy and set config parameter
"dfs.block.replicator.classname" to point to your class

On Sat, May 4, 2013 at 8:57 PM, Mohammad Mustaqeem
<3m.mustaq...@gmail.com> wrote:
> I have read somewhere that a user can specified his own 
> ReplicaPlacementPolicy.
> How can I specify my own ReplicaPlacementPolicy?
> I you have any sample ReplicaPlacementPolicy, then please share it..
>
> --
> *With regards ---*
> *Mohammad Mustaqeem*,
> M.Tech (CSE)
> MNNIT Allahabad
> 9026604270

[jira] [Created] (HDFS-4796) Port HDFS-4721 'Speed up lease/block recovery when DN fails and a block goes into recovery' to branch 1

2013-05-04 Thread Ted Yu (JIRA) 
Ted Yu created HDFS-4796:


 Summary: Port HDFS-4721 'Speed up lease/block recovery when DN 
fails and a block goes into recovery' to branch 1
 Key: HDFS-4796
 URL: https://issues.apache.org/jira/browse/HDFS-4796
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Ted Yu


This was observed while doing HBase WAL recovery. HBase uses append to write to 
its write ahead log. So initially the pipeline is setup as

DN1 --> DN2 --> DN3

This WAL needs to be read when DN1 fails since it houses the HBase regionserver 
for the WAL.

HBase first recovers the lease on the WAL file. During recovery, we choose DN1 
as the primary DN to do the recovery even though DN1 has failed and is not 
heartbeating any more.

To speedup lease/block recovery, we always choose the datanode with the most 
recent heartbeat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4797) BlockScanInfo does not override equals(..) and hashCode() consistently

2013-05-04 Thread Tsz Wo (Nicholas), SZE (JIRA) 
Tsz Wo (Nicholas), SZE created HDFS-4797:


 Summary: BlockScanInfo does not override equals(..) and hashCode() 
consistently
 Key: HDFS-4797
 URL: https://issues.apache.org/jira/browse/HDFS-4797
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: datanode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE


In the code below, equals(..) compares lastScanTime but hashCode() is computed 
using block ID.  Therefore, it could have two BlockScanInfo objects which are 
equal but have two different hash codes.
{code}
//BlockScanInfo
@Override
public int hashCode() {
  return block.hashCode();
}

@Override
public boolean equals(Object other) {
  return other instanceof BlockScanInfo &&
 compareTo((BlockScanInfo)other) == 0;
}
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-4791) Update and fix deletion of reference inode

2013-05-04 Thread Tsz Wo (Nicholas), SZE (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE resolved HDFS-4791.
--

   Resolution: Fixed
Fix Version/s: Snapshot (HDFS-2802)

I have committed this.  Thanks, Jing!

> Update and fix deletion of reference inode
> --
>
> Key: HDFS-4791
> URL: https://issues.apache.org/jira/browse/HDFS-4791
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: HDFS-4791.000.patch, HDFS-4791.001.patch, 
> HDFS-4791.002.patch, HDFS-4791.003.patch
>
>
> Currently for deleting or destroying a reference node, we directly call the 
> corresponding methods of the referred inode, and patch code for extra clean 
> after deletion. This jira will revisit the deletion of reference inodes, 
> simplify the current code and fix existing bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4798) Update computeContentSummary() for the reference nodes in snapshots

2013-05-04 Thread Tsz Wo (Nicholas), SZE (JIRA) 
Tsz Wo (Nicholas), SZE created HDFS-4798:


 Summary: Update computeContentSummary() for the reference nodes in 
snapshots
 Key: HDFS-4798
 URL: https://issues.apache.org/jira/browse/HDFS-4798
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE


Similar to quota computation, computeContentSummary() needs to be updated for 
the reference nodes in snapshots.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira