date:20121218

[jira] [Created] (HDFS-4322) Tidy up fs -help

2012-12-18 Thread Jonathan Allen (JIRA)

Jonathan Allen created HDFS-4322:


 Summary: Tidy up fs -help
 Key: HDFS-4322
 URL: https://issues.apache.org/jira/browse/HDFS-4322
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Jonathan Allen
Assignee: Jonathan Allen
Priority: Minor


There are a number of issues with the current fs -help implementation that 
could do with tidying up.

1) Running 'hadoop fs -help' displays the full detailed help for all commands.  
This produces a large amount of text and makes it hard to find what you're 
looking for.  Better functionality would be:
- hadoop fs -usage : shows the basic usage information for all commands
- hadoop fs -help : shows usage and brief summary (couple of lines) for all 
commands
- hadoop fs -help  : shows full description of the specified command

2) Help text is hardcoded into each command class. This should be replaced by 
resource files to allow internationalisation.

3) Line breaks are hardcoded into the help text. The help text should be 
auto-formatted at display time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Jenkins build became unstable: Hadoop-Hdfs-0.23-Build #467

2012-12-18 Thread Apache Jenkins Server

See

Hadoop-Hdfs-0.23-Build - Build # 467 - Unstable

2012-12-18 Thread Apache Jenkins Server

See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/467/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 12270 lines...]
[INFO] 
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @ 
hadoop-hdfs-project ---
[INFO] Wrote classpath file 
'/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/target/classes/mrapp-generated-classpath'.
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
hadoop-hdfs-project ---
[INFO] Installing 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/pom.xml
 to 
/home/jenkins/.m2/repository/org/apache/hadoop/hadoop-hdfs-project/0.23.6-SNAPSHOT/hadoop-hdfs-project-0.23.6-SNAPSHOT.pom
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-hdfs-project 
---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @ 
hadoop-hdfs-project ---
[INFO] Skipped writing classpath file 
'/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-0.23-Build/trunk/hadoop-hdfs-project/target/classes/mrapp-generated-classpath'.
  No changes found.
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.6:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is false
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  SUCCESS [5:25.614s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [42.835s]
[INFO] Apache Hadoop HDFS Project  SUCCESS [0.058s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:09.122s
[INFO] Finished at: Tue Dec 18 11:42:11 UTC 2012
[INFO] Final Memory: 52M/744M
[INFO] 
+ /home/jenkins/tools/maven/latest/bin/mvn test 
-Dmaven.test.failure.ignore=true -Pclover 
-DcloverLicenseLocation=/home/jenkins/tools/clover/latest/lib/clover.license
Archiving artifacts
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Publishing Javadoc
Recording fingerprints
Updating HDFS-4307
Updating HDFS-4315
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Unstable
Sending email for trigger: Unstable



###
## FAILED TESTS (if any) 
##
2 tests failed.
REGRESSION:  org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend

Error Message:
testComplexAppend Worker encountered exceptions.

Stack Trace:
junit.framework.AssertionFailedError: testComplexAppend Worker encountered 
exceptions.
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at 
org.apache.hadoop.hdfs.TestFileAppend2.__CLR3_0_2dvc533112i(TestFileAppend2.java:388)
at 
org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:333)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.ja

Hadoop-Hdfs-trunk - Build # 1258 - Still Failing

2012-12-18 Thread Apache Jenkins Server

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1258/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 9751 lines...]
Running org.apache.hadoop.hdfs.web.resources.TestParam
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.497 sec
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.409 sec
Running org.apache.hadoop.hdfs.web.TestOffsetUrlInputStream
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.hadoop.hdfs.web.TestWebHDFS
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 296.947 sec
Running org.apache.hadoop.hdfs.web.TestWebHdfsUrl
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.133 sec
Running org.apache.hadoop.hdfs.web.TestJsonUtil
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.167 sec
Running org.apache.hadoop.hdfs.web.TestAuthFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.379 sec
Running org.apache.hadoop.hdfs.TestConnCache
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.579 sec
Running org.apache.hadoop.hdfs.TestDFSClientRetries
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 132.858 sec
Running org.apache.hadoop.hdfs.TestListPathServlet
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.147 sec
Running org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.446 sec
Running org.apache.hadoop.hdfs.TestFileCreationEmpty
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.036 sec
Running org.apache.hadoop.hdfs.TestSetrepIncreasing

Results :

Tests run: 1112, Failures: 0, Errors: 0, Skipped: 5

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  FAILURE 
[1:38:55.513s]
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:38:56.288s
[INFO] Finished at: Tue Dec 18 13:14:50 UTC 2012
[INFO] Final Memory: 51M/484M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on 
project hadoop-hdfs: ExecutionException; nested exception is 
java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked 
VM terminated without saying properly goodbye. VM crash or System.exit called ? 
-> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Build step 'Execute shell' marked build as failure
Archiving artifacts
Updating HDFS-4140
Updating HADOOP-9004
Updating HADOOP-9140
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1258

2012-12-18 Thread Apache Jenkins Server

See 

Changes:

[atm] HDFS-4140. fuse-dfs handles open(O_TRUNC) poorly. Contributed by Colin 
Patrick McCabe.

[todd] Move HADOOP-9004 to 2.0.3 section after backport

[sradia] HADOOP-9140 Cleanup rpc PB protos (sanjay Radia)

--
[...truncated 9558 lines...]
Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.441 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.197 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.185 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.309 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.063 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.617 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.251 sec
Running org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.156 sec
Running org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.503 sec
Running org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.762 sec
Running org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.163 sec
Running org.apache.hadoop.hdfs.server.namenode.TestEditLog
Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.826 sec
Running 
org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.479 sec
Running org.apache.hadoop.hdfs.server.namenode.TestEditsDoubleBuffer
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 sec
Running org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.194 sec
Running org.apache.hadoop.hdfs.server.namenode.TestGetImageServlet
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.725 sec
Running org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.785 sec
Running org.apache.hadoop.hdfs.server.namenode.TestFSDirectory
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.553 sec
Running org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.604 sec
Running org.apache.hadoop.hdfs.server.namenode.TestFileLimit
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.985 sec
Running org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.947 sec
Running org.apache.hadoop.hdfs.server.namenode.TestNameCache
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.131 sec
Running org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionManager
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.887 sec
Running org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourcePolicy
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.281 sec
Running org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.238 sec
Running org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.864 sec
Running org.apache.hadoop.hdfs.server.namenode.TestFSImageStorageInspector
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.338 sec
Running org.apache.hadoop.hdfs.server.namenode.TestFsck
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.52 sec
Running org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.836 sec
Running org.apache.hadoop.hdfs.server.namenode.TestSecureNameNodeWithExternalKdc
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.066 sec
Running org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.727 sec
Running org.apache.hadoop.hdfs.server.namenode.TestSecureNameNode
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0

[jira] [Resolved] (HDFS-4232) NN fails to write a fsimage with stale leases

2012-12-18 Thread Daryn Sharp (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daryn Sharp resolved HDFS-4232.
---

   Resolution: Fixed
Fix Version/s: 0.23.6
   2.0.3-alpha
   3.0.0

This jira was split into subtasks, those are complete.

> NN fails to write a fsimage with stale leases
> -
>
> Key: HDFS-4232
> URL: https://issues.apache.org/jira/browse/HDFS-4232
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 0.23.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Blocker
> Fix For: 3.0.0, 2.0.3-alpha, 0.23.6
>
> Attachments: HDFS-4232.patch
>
>
> The reading of a fsimage will ignore leases for non-existent files, but the 
> writing of an image will fail if there are leases for non-existent files.  If 
> the image contains leases that reference a non-existent file, then the NN 
> will fail to start, and the 2NN will start but fail to ever write an image.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4323) HttpFS is using Tomcat 6.0.32 which has numerous vulnerabilities

2012-12-18 Thread Jeff Lord (JIRA)

Jeff Lord created HDFS-4323:
---

 Summary: HttpFS is using Tomcat 6.0.32 which has numerous 
vulnerabilities
 Key: HDFS-4323
 URL: https://issues.apache.org/jira/browse/HDFS-4323
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 2.0.2-alpha
Reporter: Jeff Lord


Apache Tomcat DIGEST Vulnerabilities Medium CVE-2011-5064 5.0 Fail 
Apache Tomcat DIGEST Vulnerabilities Medium CVE-2011-5063 5.0 Fail  
Apache Tomcat DIGEST Vulnerabilities Medium CVE-2011-1184 5.0 Fail  
Apache Tomcat DIGEST Vulnerabilities Medium CVE-2011-5062 5.0 Fail  

Apache Tomcat Multiple Vulnerabilities (201108) Medium CVE-2011-2204 5.0 Fail 
Apache Tomcat Multiple Vulnerabilities (201108) Medium CVE-2011-2526 5.0 Fail 
Apache Tomcat Multiple Vulnerabilities (201108) Medium CVE-2011-2481 5.0 Fail 
Apache Tomcat Multiple Vulnerabilities (201108) Medium CVE-2011-2729 5.0 Fail 

Apache Tomcat Request Object Information Disclosure (20111001) Medium 
CVE-2011-3375 5.0 Fail

All of these have been fixed in subsequent release of Tomcat.
http://tomcat.apache.org/security-6.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4324) Track and report out-of-date blocks separately from corrupt blocks

2012-12-18 Thread Todd Lipcon (JIRA)

Todd Lipcon created HDFS-4324:
-

 Summary: Track and report out-of-date blocks separately from 
corrupt blocks
 Key: HDFS-4324
 URL: https://issues.apache.org/jira/browse/HDFS-4324
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: namenode
Affects Versions: 3.0.0
Reporter: Todd Lipcon


Currently in various places (metrics, dfsadmin -report, fsck, logs) we use the 
term "corrupt" to refer to blocks which have an out-of-date generation stamp. 
Since out-of-date blocks are a fairly normal occurrence if a DN restarts while 
data is being written, we should be avoid using 'scary' works like _corrupt_. 
This may need both some textual changes as well as some internal changes to 
count the corruption types distinctly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4322) Tidy up fs -help

Jenkins build became unstable: Hadoop-Hdfs-0.23-Build #467

Hadoop-Hdfs-0.23-Build - Build # 467 - Unstable

Hadoop-Hdfs-trunk - Build # 1258 - Still Failing

Build failed in Jenkins: Hadoop-Hdfs-trunk #1258

[jira] [Resolved] (HDFS-4232) NN fails to write a fsimage with stale leases

[jira] [Created] (HDFS-4323) HttpFS is using Tomcat 6.0.32 which has numerous vulnerabilities

[jira] [Created] (HDFS-4324) Track and report out-of-date blocks separately from corrupt blocks

8 matches

Site Navigation

Mail list logo

Footer information