Re: Why could not find finished jobs in yarn.resourcemanager.webapp.address?

Can anyone help this issue? Thanks! 2013/5/2 sam liu > Hi, > > I launched yarn and its webapp on port 18088, and then successfully > launched and executed some test MR jobs like 'hadoop jar > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.3-alpha.jar pi 2 30'. > > But, when login the web

[jira] [Resolved] (HDFS-4782) backport edit log corruption toleration to branch-1-win

[ https://issues.apache.org/jira/browse/HDFS-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-4782. -- Resolution: Fixed Fix Version/s: 1-win I have committed this. Thanks, Ch

Re: Heads up - 2.0.5-beta

On Thu, May 2, 2013 at 2:11 AM, Konstantin Shvachko wrote: > On Thu, May 2, 2013 at 12:07 AM, Chris Douglas wrote: >> Can anyone remember why we vote on release plans? -C > > To vote on features to include in the release. Since most features are developed in branches (requiring a merge vote), ea

[jira] [Created] (HDFS-4793) uploading file larger than the spaceQuota limit should not create 0 byte file

yeshavora created HDFS-4793: --- Summary: uploading file larger than the spaceQuota limit should not create 0 byte file Key: HDFS-4793 URL: https://issues.apache.org/jira/browse/HDFS-4793 Project: Hadoop HDFS

Fwd: Heads up - 2.0.5-beta

Konstantin, I am arguing against invasive and destructive features proposed for the > release. > Your choice of words is deplorable, to say the least. Can you explain what do you mean by *destructive*? Please substantiate your claim on technical grounds. So far you have been quiet while we have

[jira] [Created] (HDFS-4792) remove unused import from TestEditLog in branch-1

Chris Nauroth created HDFS-4792: --- Summary: remove unused import from TestEditLog in branch-1 Key: HDFS-4792 URL: https://issues.apache.org/jira/browse/HDFS-4792 Project: Hadoop HDFS Issue Type:

Re: Heads up - 2.0.5-beta

Konstantin, On May 2, 2013, at 2:08 AM, Konstantin Shvachko wrote: > I am arguing against invasive and destructive features proposed for the > release. > Just to remind here they are again, since the history has been wiped out. > > # Snapshots > # NFS gateway for HDFS > # HDFS-347 unix domain so

[jira] [Created] (HDFS-4791) Update and fix deletion of reference inode

Jing Zhao created HDFS-4791: --- Summary: Update and fix deletion of reference inode Key: HDFS-4791 URL: https://issues.apache.org/jira/browse/HDFS-4791 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Resolved] (HDFS-4774) Backport HDFS-4525 'Provide an API for knowing whether file is closed or not' to branch-1

[ https://issues.apache.org/jira/browse/HDFS-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-4774. -- Resolution: Fixed Fix Version/s: 1.2.0 I have committed this. Thanks, Te

[jira] [Created] (HDFS-4790) TestFsShellReturnCode may fail in branch-1

Tsz Wo (Nicholas), SZE created HDFS-4790: Summary: TestFsShellReturnCode may fail in branch-1 Key: HDFS-4790 URL: https://issues.apache.org/jira/browse/HDFS-4790 Project: Hadoop HDFS

Re: Is Hadoop SequenceFile binary safe?

It seems like we could just set up an escape sequence and make it actually binary-safe, rather than just probabilistically. The escape sequence would only be inserted when there would otherwise be confusion between data and a sync marker. best, Colin On Thu, May 2, 2013 at 3:26 AM, Hs wrote:

[jira] [Created] (HDFS-4789) redirectToRandomDataNode tries to get a delegation token without verifying security is enabled

Stephen Fritz created HDFS-4789: --- Summary: redirectToRandomDataNode tries to get a delegation token without verifying security is enabled Key: HDFS-4789 URL: https://issues.apache.org/jira/browse/HDFS-4789

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

Additional reason, HDFS does not have limit on number of files in a directory. Some clusters had millions of files in a single directory. Listing such a directory resulted in very large responses, requiring large contiguous memory allocation in JVM (for the array) and unpredictable GC failures. O

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

Hi Brad, The reasoning is that the NameNode locking is somewhat coarse grained. In older versions of Hadoop, before it worked this way, we found that listing large directories (eg with 100k+ files) could end up holding the namenode's lock for a quite long period of time and starve other clients.

DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

Could someone explain why the DistributedFileSystem's listStatus() method does a piecemeal assembly of a directory listing within the method? Is there a locking issue? What if an element is added to the the directory during the operation?  What if elements are removed? It would make sense to me

Build failed in Jenkins: Hadoop-Hdfs-trunk #1390

See Changes: [suresh] HDFS-4784. NPE in FSDirectory.resolvePath(). Contributed by Brandon Li. [suresh] HDFS-4785. Concat operation does not remove concatenated files from InodeMap. Contributed by Suresh Srinivas. [atm] MAPREDUCE-51

Hadoop-Hdfs-trunk - Build # 1390 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1390/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10251 lines...] Running org.apache.hadoop.fs.viewfs.TestV

Jenkins build is back to stable : Hadoop-Hdfs-0.23-Build #599

See

[jira] [Created] (HDFS-4788) Fix bug in bestNode function which caused 'Could not reach the block' exception even when there're nodes available

Yuyang Lan created HDFS-4788: Summary: Fix bug in bestNode function which caused 'Could not reach the block' exception even when there're nodes available Key: HDFS-4788 URL: https://issues.apache.org/jira/browse/HDFS-

Re: Is Hadoop SequenceFile binary safe?

Hi Chris, thanks for your replay. That's to say, SequenceFile is probabilistically binary safe. I notice a jira issue attempting to support "append" in existing SequenceFile( https://issues.apache.org/jira/browse/HADOOP-7139). It occurred to me that if some hacker reads the sync marker from the ex

Re: Heads up - 2.0.5-beta

On Thu, May 2, 2013 at 12:07 AM, Chris Douglas wrote: > Can anyone remember why we vote on release plans? -C To vote on features to include in the release. Thanks, --Konstantin

Re: Heads up - 2.0.5-beta

I am not sure what was your point here. You seem to be assuming things I never mentioned. I am arguing against invasive and destructive features proposed for the release. Just to remind here they are again, since the history has been wiped out. # Snapshots # NFS gateway for HDFS # HDFS-347 unix d

[jira] [Created] (HDFS-4787) Create a new HdfsConfiguration before each TestDFSClientRetries testcases

Tian Hong Wang created HDFS-4787: Summary: Create a new HdfsConfiguration before each TestDFSClientRetries testcases Key: HDFS-4787 URL: https://issues.apache.org/jira/browse/HDFS-4787 Project: Hadoop

Re: Heads up - 2.0.5-beta

On Wed, May 1, 2013 at 6:24 PM, Arun C Murthy wrote: > Having a strict policy leads to all sorts of further dialogues and issues we > could do well without. +1 Can anyone remember why we vote on release plans? -C