Hadoop-Hdfs-trunk - Build # 1202 - Failure

2012-10-21 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1202/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 11879 lines...]
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.372 sec
Running org.apache.hadoop.fs.viewfs.TestViewFsHdfs
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.792 sec
Running org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.098 sec
Running org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.741 sec
Running org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.771 sec
Running org.apache.hadoop.fs.permission.TestStickyBit
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.884 sec
Running org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.963 sec
Running org.apache.hadoop.fs.TestFcHdfsSymlink
Tests run: 68, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.129 sec
Running org.apache.hadoop.fs.TestFcHdfsCreateMkdir
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.143 sec
Running org.apache.hadoop.fs.TestHDFSFileContextMainOperations
Tests run: 59, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.125 sec

Results :

Tests in error: 
  
testSetrepIncWithUnderReplicatedBlocks(org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks):
 test timed out after 30 milliseconds

Tests run: 1607, Failures: 0, Errors: 1, Skipped: 4

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  FAILURE 
[1:22:38.262s]
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:22:39.039s
[INFO] Finished at: Sun Oct 21 12:56:27 UTC 2012
[INFO] Final Memory: 19M/464M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Build step 'Execute shell' marked build as failure
Archiving artifacts
Updating HDFS-4022
Updating HADOOP-8932
Updating HADOOP-8923
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1202

2012-10-21 Thread Apache Jenkins Server 
See 

Changes:

[umamahesh] HDFS-4022. Replication not happening for appended block. 
Contributed by Vinay.

[suresh] Change incorrect jira number HADOOP-8923 to HADOOP-8932 introduced in 
commit r1398883.

--
[...truncated 11686 lines...]
Running org.apache.hadoop.hdfs.TestHftpFileSystem
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.997 sec
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.86 sec
Running org.apache.hadoop.hdfs.TestFSInputChecker
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.178 sec
Running org.apache.hadoop.hdfs.TestHftpDelegationToken
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.677 sec
Running org.apache.hadoop.hdfs.TestDFSRollback
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.925 sec
Running org.apache.hadoop.hdfs.TestByteRangeInputStream
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.308 sec
Running org.apache.hadoop.hdfs.TestPersistBlocks
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.607 sec
Running org.apache.hadoop.hdfs.TestRenameWhileOpen
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.696 sec
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.427 sec
Running org.apache.hadoop.hdfs.TestDatanodeDeath
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 137.933 sec
Running org.apache.hadoop.hdfs.TestLeaseRecovery
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.588 sec
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.214 sec
Running org.apache.hadoop.hdfs.protocol.datatransfer.TestPacketReceiver
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.349 sec
Running org.apache.hadoop.hdfs.protocol.TestLayoutVersion
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 sec
Running org.apache.hadoop.hdfs.protocol.TestExtendedBlock
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.055 sec
Running org.apache.hadoop.hdfs.TestHDFSServerPorts
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.082 sec
Running org.apache.hadoop.hdfs.TestFileCreationDelete
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.471 sec
Running org.apache.hadoop.hdfs.TestDFSMkdirs
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.543 sec
Running org.apache.hadoop.hdfs.TestDistributedFileSystem
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.454 sec
Running org.apache.hadoop.hdfs.TestDecommission
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 119.89 sec
Running org.apache.hadoop.hdfs.TestLeaseRecovery2
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.308 sec
Running org.apache.hadoop.hdfs.TestFileStatus
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.246 sec
Running org.apache.hadoop.hdfs.TestBlockMissingException
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.921 sec
Running org.apache.hadoop.hdfs.TestDataTransferKeepalive
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.633 sec
Running org.apache.hadoop.hdfs.TestLeaseRenewer
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.933 sec
Running org.apache.hadoop.hdfs.TestFileAppend
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.644 sec
Running org.apache.hadoop.hdfs.TestDatanodeConfig
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.225 sec
Running org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.3 sec
Running org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
Tests run: 49, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.154 sec
Running org.apache.hadoop.hdfs.web.resources.TestParam
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.392 sec
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.384 sec
Running org.apache.hadoop.hdfs.web.TestOffsetUrlInputStream
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.hadoop.hdfs.web.TestWebHDFS
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.581 sec
Running org.apache.hadoop.hdfs.web.TestWebHdfsUrl
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.047 sec
Running org.apache.hadoop.hdfs.web.TestJsonUtil
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.166 sec
Running org.apache.hadoop.hdfs.web.TestAuthFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.288 sec
Running org.apache.hadoop.hdfs.TestConnCache

[jira] [Resolved] (HDFS-4087) Protocol changes for listSnapshots functionality

2012-10-21 Thread Suresh Srinivas (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas resolved HDFS-4087.
---

   Resolution: Fixed
Fix Version/s: Snapshot (HDFS-2802)
 Hadoop Flags: Reviewed

I committed the patch. Thank you Brandon.

> Protocol changes for listSnapshots functionality
> 
>
> Key: HDFS-4087
> URL: https://issues.apache.org/jira/browse/HDFS-4087
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: name-node
>Affects Versions: Snapshot (HDFS-2802)
>Reporter: Brandon Li
>Assignee: Brandon Li
>  Labels: needs-test
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: HDFS-4087.patch, HDFS-4087.patch, HDFS-4087.patch, 
> HDFS-4087.patch
>
>
> SnapInfo saves information about a snapshot. This jira also updates the java 
> protocol classes and translation for listSnapshot operation.
> Given a snapshot root, the snapshots create under it can be listed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-4079) Add SnapshotManager

2012-10-21 Thread Tsz Wo (Nicholas), SZE (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE resolved HDFS-4079.
--

   Resolution: Fixed
Fix Version/s: Snapshot (HDFS-2802)
 Hadoop Flags: Reviewed

I have committed this.

> Add SnapshotManager
> ---
>
> Key: HDFS-4079
> URL: https://issues.apache.org/jira/browse/HDFS-4079
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: name-node
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: h4079_20121019.patch
>
>
> SnapshotManager maintains a list for all the snapshottable directories in the 
> namespace.  It also supports snapshot related methods such as setting a 
> directory to snapshottable, creating a snapshot, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4098) Support append to snapshotted files

2012-10-21 Thread Tsz Wo (Nicholas), SZE (JIRA) 
Tsz Wo (Nicholas), SZE created HDFS-4098:


 Summary: Support append to snapshotted files
 Key: HDFS-4098
 URL: https://issues.apache.org/jira/browse/HDFS-4098
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: name-node
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE


When a regular file is reopened for append, the type is changed from INodeFile 
to INodeFileUnderConstruction.  The type of snapshotted files (i.e. original 
files) is INodeFileWithLink.  We have to support similar "under construction" 
INodeFileWithLink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-4078) Handle replication in snapshots

2012-10-21 Thread Tsz Wo (Nicholas), SZE (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE resolved HDFS-4078.
--

   Resolution: Fixed
Fix Version/s: Snapshot (HDFS-2802)
 Hadoop Flags: Reviewed

Since trunk does not yet have snapshot, file replication and block replication 
are the same.  Let's change trunk when we merge the branch.

I have commit this.

> Handle replication in snapshots
> ---
>
> Key: HDFS-4078
> URL: https://issues.apache.org/jira/browse/HDFS-4078
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: name-node
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>  Labels: needs-test
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: h4078_20121021.patch
>
>
> Without snapshots, file replication is the same as block replication.
> With snapshot, the file replication R_o of the original file and the file 
> replication R_s of the snapshot file could possibly be different.  Since the 
> blocks are shared between the original file and the snapshot file, block 
> replication is max(R_o, R_s).  If there are more than one snapshots, block 
> replication is the max file replication of the original file and all snapshot 
> files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4099) Clean up replication code and add more javadoc

2012-10-21 Thread Tsz Wo (Nicholas), SZE (JIRA) 
Tsz Wo (Nicholas), SZE created HDFS-4099:


 Summary: Clean up replication code and add more javadoc
 Key: HDFS-4099
 URL: https://issues.apache.org/jira/browse/HDFS-4099
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE


- FSNamesystem.checkReplicationFactor(..) should be combined with 
BlockManager.checkReplication(..).
- Add javadoc to the replication related method in BlockManager.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

How to speedup test case running?

2012-10-21 Thread 谢良 
Hi devs,

are there any tips or parameters to pass to "mvn test" to make the running more 
aggressively?  it costs me almost two hours for a "mvn test" run under 
"haddop-hdfs" directory.  I tried the magic in HBase community(ramdisk & 
surefire.secondPartThreadCount),  seems didn't work here:)

Best,
Liang