Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

I think this is a great work, Todd. And I think we should not merge it into trunk or other branches. As I suggested earlier on this list I think this should be spinned off as a separate project or a subproject. - The code is well detached as a self contained package. - It is a logically stand-alon

[jira] [Created] (HDFS-3979) Fix hsync and hflush semantics.

Lars Hofhansl created HDFS-3979: --- Summary: Fix hsync and hflush semantics. Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug R

Re: Commits breaking compilation of MR 'classic' tests

How about adding this step to the MR PreCommit jenkins job so it's run as part test-patch? On Tue, Sep 25, 2012 at 7:48 PM, Arun C Murthy wrote: > Committers, > > As most people are aware, the MapReduce 'classic' tests (in > hadoop-mapreduce-project/src/test) still need to built using ant since

Re: need some help : compiling latest hadoop from source

Hey Sujee, Yes. During the course of 0.21 and 0.22, the project repos underwent a split at the global repo level. For 0.23 (and thereby 2.x) we've gone back to hosting projects under the single, common repository itself (as was in past). On Wed, Sep 26, 2012 at 4:46 AM, Sujee Maniyam wrote: > th

Commits breaking compilation of MR 'classic' tests

Committers, As most people are aware, the MapReduce 'classic' tests (in hadoop-mapreduce-project/src/test) still need to built using ant since they aren't mavenized yet. I've seen several commits (and 2 within the last hour i.e. MAPREDUCE-3681 and MAPREDUCE-3682) which lead me to believe dev

[jira] [Reopened] (HDFS-1322) DistributedFileSystem.mkdirs(dir, dirPermission) doesn't set the permissions of created dir to dirPermission

[ https://issues.apache.org/jira/browse/HDFS-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reopened HDFS-1322: Assignee: Colin Patrick McCabe (was: Anwar Abdus-Samad) reopening as a doc change

[jira] [Resolved] (HDFS-2798) Append may race with datanode block scanner, causing replica to be incorrectly marked corrupt

[ https://issues.apache.org/jira/browse/HDFS-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li resolved HDFS-2798. -- Resolution: Duplicate Target Version/s: 2.0.0-alpha, 0.23.3, 3.0.0 (was: 0.23.3, 2.0.0-alpha,

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

+1 Awesome work Todd. On Tue, Sep 25, 2012 at 4:02 PM, Todd Lipcon wrote: > Dear fellow HDFS developers, > > Per my email thread last week ("Heads up: merge for QJM branch soon" > at http://markmail.org/message/vkyh5culdsuxdb6t) I would like to > propose merging the HDFS-3077 branch into trunk.

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

On Tue, Sep 25, 2012 at 4:02 PM, Todd Lipcon wrote: > detail in a document attached to HDFS-3077 itself. The code itself has > been contributed by me, Aaron, and Eli, but I'd be remiss not to also I should actually amend the above: I had forgotten that some of the early patches on the branch were

Re: need some help : compiling latest hadoop from source

thank you! another thing that is puzzling me, within hadoop-common repo, I see 'hadoop-mapreduce-project' and 'hadoop-hdfs-project'. But I also see 'hadoop-hdfs' and 'hadoop-mapreduce' listed as separate repositories here : http://git.apache.org/ is 'hadoop-common' the encompassing project no

Re: need some help : compiling latest hadoop from source

You're doing the right thing. The ./hadoop-dist/target/hadoop-3.0.0-SNAPSHOT.tar.gz is the trunk tarball you'll want to deploy. On Wed, Sep 26, 2012 at 3:58 AM, Sujee Maniyam wrote: > HI > I am following this > http://www.michael-noll.com/blog/2011/04/14/building-an-hadoop-0-20-x-version-for-hbas

[VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

Dear fellow HDFS developers, Per my email thread last week ("Heads up: merge for QJM branch soon" at http://markmail.org/message/vkyh5culdsuxdb6t) I would like to propose merging the HDFS-3077 branch into trunk. The branch has been active since mid July and has stabilized significantly over the la

[jira] [Created] (HDFS-3977) Incompatible change between hadoop-1 and hadoop-2 when the dfs.hosts and dfs.hosts.exclude files are not present

Arpit Gupta created HDFS-3977: - Summary: Incompatible change between hadoop-1 and hadoop-2 when the dfs.hosts and dfs.hosts.exclude files are not present Key: HDFS-3977 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Created] (HDFS-3976) SampleQuantiles#query is O(N^2) instead of O(N)

Andrew Wang created HDFS-3976: - Summary: SampleQuantiles#query is O(N^2) instead of O(N) Key: HDFS-3976 URL: https://issues.apache.org/jira/browse/HDFS-3976 Project: Hadoop HDFS Issue Type: Impro

Re: [Updated] RE: current direction in namenode HA

The quorum-based journal implementation is in HDFS-3077, soon to be merged to trunk. The BookKeeper-based implementation has been developed on trunk under a contrib/ module. Thanks -Todd On Tue, Sep 25, 2012 at 3:34 PM, Sujee Maniyam wrote: > and what branch is this work going into? > thanks >

Re: [Updated] RE: current direction in namenode HA

and what branch is this work going into? thanks Sujee On Mon, Sep 10, 2012 at 4:18 PM, Sujee Maniyam wrote: > Thanks very much for all your input! Much appreciated... >

need some help : compiling latest hadoop from source

HI I am following this http://www.michael-noll.com/blog/2011/04/14/building-an-hadoop-0-20-x-version-for-hbase-0-90-2/ How ever the latest hadoop is using mvn. Just wanted to verify that I am doing the right thing The following completed successfully. # git clone http://git.apache.org/hadoop-co

[jira] [Created] (HDFS-3975) Add a test for umask in webhdfs

Colin Patrick McCabe created HDFS-3975: -- Summary: Add a test for umask in webhdfs Key: HDFS-3975 URL: https://issues.apache.org/jira/browse/HDFS-3975 Project: Hadoop HDFS Issue Type: Tes

[jira] [Resolved] (HDFS-1322) DistributedFileSystem.mkdirs(dir, dirPermission) doesn't set the permissions of created dir to dirPermission

[ https://issues.apache.org/jira/browse/HDFS-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-1322. Resolution: Invalid As Suresh commented, this is working as designed. If you don't

[jira] [Created] (HDFS-3974) dfsadmin -metasave throws NPE when under-replicated blocks are recently deleted

Todd Lipcon created HDFS-3974: - Summary: dfsadmin -metasave throws NPE when under-replicated blocks are recently deleted Key: HDFS-3974 URL: https://issues.apache.org/jira/browse/HDFS-3974 Project: Hadoop

[jira] [Created] (HDFS-3973) Old trash directories are never deleted on upgrade from 1.x

Robert Joseph Evans created HDFS-3973: - Summary: Old trash directories are never deleted on upgrade from 1.x Key: HDFS-3973 URL: https://issues.apache.org/jira/browse/HDFS-3973 Project: Hadoop HDF

[jira] [Created] (HDFS-3972) Trash emptier fails in secure cluster

Todd Lipcon created HDFS-3972: - Summary: Trash emptier fails in secure cluster Key: HDFS-3972 URL: https://issues.apache.org/jira/browse/HDFS-3972 Project: Hadoop HDFS Issue Type: Bug C

[jira] [Created] (HDFS-3971) Add a resume feature to the copyFromLocal and put commands

Adam Muise created HDFS-3971: Summary: Add a resume feature to the copyFromLocal and put commands Key: HDFS-3971 URL: https://issues.apache.org/jira/browse/HDFS-3971 Project: Hadoop HDFS Issue Ty

[jira] [Created] (HDFS-3970) BlockPoolSliceStorage#doRollback(..) should use BlockPoolSliceStorage instead of DataStorage to read prev version file.

Vinay created HDFS-3970: --- Summary: BlockPoolSliceStorage#doRollback(..) should use BlockPoolSliceStorage instead of DataStorage to read prev version file. Key: HDFS-3970 URL: https://issues.apache.org/jira/browse/HDFS-3970

Hadoop-Hdfs-0.23-Build - Build # 385 - Still Unstable

See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/385/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 18920 lines...] [INFO] Executed tasks [INFO] [INFO]

Jenkins build is still unstable: Hadoop-Hdfs-0.23-Build #385

See

[jira] [Resolved] (HDFS-3347) NullPointerException When trying to log to editstreams

[ https://issues.apache.org/jira/browse/HDFS-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay resolved HDFS-3347. - Resolution: Duplicate Already fixed in HDFS-3026, > NullPointerException When trying to log to ed