Jenkins build is back to stable : Hadoop-Hdfs-0.23-Build #413

2012-10-23 Thread Apache Jenkins Server 
See

[jira] [Resolved] (HDFS-4084) provide CLI support for allow and disallow snapshot on a directory

2012-10-23 Thread Suresh Srinivas (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas resolved HDFS-4084.
---

   Resolution: Fixed
Fix Version/s: Snapshot (HDFS-2802)
 Hadoop Flags: Reviewed

I committed the patch to HDFS-2802 branch. Thank you Brandon.

> provide CLI support for allow and disallow snapshot on a directory
> --
>
> Key: HDFS-4084
> URL: https://issues.apache.org/jira/browse/HDFS-4084
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: hdfs client, name-node, tools
>Affects Versions: Snapshot (HDFS-2802)
>Reporter: Brandon Li
>Assignee: Brandon Li
>  Labels: needs-test
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: HDFS-4084.patch, HDFS-4084.patch, HDFS-4084.patch
>
>
> To provide CLI support to allow snapshot, disallow snapshot on a directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

need some help understanding NN quorum edit log

2012-10-23 Thread Sujee Maniyam 
Hi devs,
I am reading through the HA code, trying to understand how the
editlogs propagate.

I have looked at the following classes
EditLogTailer
JournalNode
QuorumJournalManager
NameNode
FSNameSystem

I see FSNameSystem is using FSImage editlog.   where is the tie-in
between  NN and  QuorumJournal?

thanks
Sujee Maniyam

http://sujee.net

Re: need some help understanding NN quorum edit log

2012-10-23 Thread Todd Lipcon 
Hi Sujee,

QuorumJournalManager implements the JournalManager interface. The javadoc
on JournalManager may help you understand. Also, I would recommend
understanding the local-disk implementation (FileJournalManager) before
digging too deep into the QuorumJournalManager, which is a bit more complex.

-Todd

On Tue, Oct 23, 2012 at 3:31 PM, Sujee Maniyam  wrote:

> Hi devs,
> I am reading through the HA code, trying to understand how the
> editlogs propagate.
>
> I have looked at the following classes
> EditLogTailer
> JournalNode
> QuorumJournalManager
> NameNode
> FSNameSystem
>
> I see FSNameSystem is using FSImage editlog.   where is the tie-in
> between  NN and  QuorumJournal?
>
> thanks
> Sujee Maniyam
>
> http://sujee.net
>



-- 
Todd Lipcon
Software Engineer, Cloudera

[jira] [Resolved] (HDFS-4093) In branch-1-win, AzureBlockPlacementPolicy#chooseTarget only returns one DN when replication factor is greater than 3.

2012-10-23 Thread Tsz Wo (Nicholas), SZE (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE resolved HDFS-4093.
--

   Resolution: Fixed
Fix Version/s: 1-win

I have committed this.  Thanks, Jing!

> In branch-1-win, AzureBlockPlacementPolicy#chooseTarget only returns one DN 
> when replication factor is greater than 3. 
> ---
>
> Key: HDFS-4093
> URL: https://issues.apache.org/jira/browse/HDFS-4093
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Fix For: 1-win
>
> Attachments: HDFS-b1-win-4093.001.patch, HDFS-b1-win-4093.002.patch
>
>
> In branch-1-win, when AzureBlockPlacementPolicy (which extends the 
> BlockPlacementPolicyDefault) is used, if the client increases the number of 
> replicas (e.g., from 3 to 10), AzureBlockPlacementPolicy#chooseTarget will 
> return only 1 Datanode each time. Thus in 
> FSNameSystem#computeReplicationWorkForBlock, it is possible that the 
> replication monitor may choose a datanode that has been chosen as target but 
> still in the pendingReplications (because computeReplicationWorkForBlock does 
> not check the pending replication before doing the chooseTarget). 
> To avoid this "hit-the-same-datanode" scenario, we modify the 
> AzureBlockPlacementPolicy#chooseTarget to make it return multiple DN. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira