See <https://builds.apache.org/job/Hadoop-Hdfs-trunk/3030/changes>
Changes: [jing9] HDFS-10216. Distcp -diff throws exception when handling relative path. [kihwal] HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by [xyao] HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. [wangda] MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some ------------------------------------------ [...truncated 27065 lines...] Running org.apache.hadoop.hdfs.TestDFSUtil Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.758 sec - in org.apache.hadoop.hdfs.TestDFSUtil Running org.apache.hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.985 sec - in org.apache.hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs Running org.apache.hadoop.hdfs.TestExternalBlockReader Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.796 sec - in org.apache.hadoop.hdfs.TestExternalBlockReader Running org.apache.hadoop.hdfs.TestDeprecatedKeys Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.753 sec - in org.apache.hadoop.hdfs.TestDeprecatedKeys Running org.apache.hadoop.hdfs.TestGetBlocks Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.003 sec - in org.apache.hadoop.hdfs.TestGetBlocks Running org.apache.hadoop.hdfs.TestRead Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.964 sec - in org.apache.hadoop.hdfs.TestRead Running org.apache.hadoop.hdfs.TestParallelReadUtil Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.042 sec - in org.apache.hadoop.hdfs.TestParallelReadUtil Running org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.947 sec - in org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache Running org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead Tests run: 11, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 14.126 sec <<< FAILURE! - in org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead testSmallFileLocalRead(org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead) Time elapsed: 1.05 sec <<< ERROR! java.lang.IndexOutOfBoundsException: Requested more bytes than destination buffer size at org.apache.hadoop.fs.FSInputStream.validatePositionedReadArgs(FSInputStream.java:107) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:975) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.checkFileContent(TestShortCircuitLocalRead.java:157) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadImpl(TestShortCircuitLocalRead.java:286) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitRead(TestShortCircuitLocalRead.java:241) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.testSmallFileLocalRead(TestShortCircuitLocalRead.java:308) testLocalReadLegacy(org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead) Time elapsed: 0.49 sec <<< ERROR! java.lang.IndexOutOfBoundsException: Requested more bytes than destination buffer size at org.apache.hadoop.fs.FSInputStream.validatePositionedReadArgs(FSInputStream.java:107) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:975) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.checkFileContent(TestShortCircuitLocalRead.java:157) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadImpl(TestShortCircuitLocalRead.java:286) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadLegacy(TestShortCircuitLocalRead.java:235) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.testLocalReadLegacy(TestShortCircuitLocalRead.java:316) testLocalReadFallback(org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead) Time elapsed: 0.666 sec <<< ERROR! java.lang.IndexOutOfBoundsException: Requested more bytes than destination buffer size at org.apache.hadoop.fs.FSInputStream.validatePositionedReadArgs(FSInputStream.java:107) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:975) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.checkFileContent(TestShortCircuitLocalRead.java:157) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadImpl(TestShortCircuitLocalRead.java:286) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadLegacy(TestShortCircuitLocalRead.java:235) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.testLocalReadFallback(TestShortCircuitLocalRead.java:327) Running org.apache.hadoop.hdfs.TestMultiThreadedHflush Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.582 sec - in org.apache.hadoop.hdfs.TestMultiThreadedHflush Running org.apache.hadoop.hdfs.TestReconstructStripedFile Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.14 sec - in org.apache.hadoop.hdfs.TestReconstructStripedFile Running org.apache.hadoop.hdfs.TestSafeModeWithStripedFile Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.916 sec - in org.apache.hadoop.hdfs.TestSafeModeWithStripedFile Running org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 73.756 sec - in org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.677 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart testFileLengthWithHSyncAndClusterRestartWithOutDNsRegister(org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart) Time elapsed: 2.584 sec <<< ERROR! java.io.IOException: Failed to save in any storage directories while saving namespace. at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1129) at org.apache.hadoop.hdfs.server.namenode.FSImage.saveNamespace(FSImage.java:1055) at org.apache.hadoop.hdfs.server.namenode.FSImage.saveNamespace(FSImage.java:1025) at org.apache.hadoop.hdfs.server.namenode.FSImage.saveNamespace(FSImage.java:1030) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:998) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:658) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:640) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:702) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:903) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:882) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1599) at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1247) at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1016) at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:891) at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:823) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:482) at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:441) at org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart.testFileLengthWithHSyncAndClusterRestartWithOutDNsRegister(TestFileLengthOnClusterRestart.java:42) Running org.apache.hadoop.hdfs.TestRemoteBlockReader2 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.135 sec - in org.apache.hadoop.hdfs.TestRemoteBlockReader2 Running org.apache.hadoop.hdfs.TestBlockMissingException Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.517 sec - in org.apache.hadoop.hdfs.TestBlockMissingException Running org.apache.hadoop.hdfs.TestReplication Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 52.156 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestReplication testBadBlockReportOnTransferMissingBlockFile(org.apache.hadoop.hdfs.TestReplication) Time elapsed: 0.504 sec <<< ERROR! java.lang.IllegalArgumentException: null at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImplTestUtils.<init>(FsDatasetImplTestUtils.java:201) at org.apache.hadoop.hdfs.server.datanode.FsDatasetImplTestUtilsFactory.newInstance(FsDatasetImplTestUtilsFactory.java:30) at org.apache.hadoop.hdfs.MiniDFSCluster.getFsDatasetTestUtils(MiniDFSCluster.java:1842) at org.apache.hadoop.hdfs.MiniDFSCluster.corruptBlockOnDataNodesHelper(MiniDFSCluster.java:2035) at org.apache.hadoop.hdfs.MiniDFSCluster.corruptBlockOnDataNodesByDeletingBlockFile(MiniDFSCluster.java:2068) at org.apache.hadoop.hdfs.TestReplication.testBadBlockReportOnTransfer(TestReplication.java:179) at org.apache.hadoop.hdfs.TestReplication.testBadBlockReportOnTransferMissingBlockFile(TestReplication.java:222) Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.338 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 Running org.apache.hadoop.hdfs.TestFileAppend Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.761 sec - in org.apache.hadoop.hdfs.TestFileAppend Running org.apache.hadoop.hdfs.TestDistributedFileSystem Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.865 sec - in org.apache.hadoop.hdfs.TestDistributedFileSystem Running org.apache.hadoop.hdfs.TestClientBlockVerification Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.399 sec - in org.apache.hadoop.hdfs.TestClientBlockVerification Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.357 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 Running org.apache.hadoop.hdfs.TestReservedRawPaths Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.759 sec - in org.apache.hadoop.hdfs.TestReservedRawPaths Running org.apache.hadoop.hdfs.TestMissingBlocksAlert Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.549 sec - in org.apache.hadoop.hdfs.TestMissingBlocksAlert Running org.apache.hadoop.hdfs.TestClose Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.555 sec - in org.apache.hadoop.hdfs.TestClose Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.889 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 Running org.apache.hadoop.hdfs.TestFileConcurrentReader Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.848 sec - in org.apache.hadoop.hdfs.TestFileConcurrentReader Running org.apache.hadoop.hdfs.TestHDFSServerPorts Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.664 sec - in org.apache.hadoop.hdfs.TestHDFSServerPorts Running org.apache.hadoop.tracing.TestTracing Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.76 sec - in org.apache.hadoop.tracing.TestTracing Running org.apache.hadoop.tracing.TestTraceAdmin Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.76 sec - in org.apache.hadoop.tracing.TestTraceAdmin Running org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.989 sec - in org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead Results : Failed tests: TestDFSClientExcludedNodes.testExcludedNodesForgiveness:148 Excluded DataNodes should be forgiven after a while and not cause file writing exception of: 'File /testForgivingExcludedNodes could only be replicated to 0 nodes instead of minReplication (=1). There are 3 datanode(s) running and 3 node(s) are excluded in this operation. at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1764) at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:302) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2494) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:839) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:508) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2423) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2419) ' Tests in error: TestReadStripedFileWithDecoding>Object.wait:-2 » test timed out after 300000 ... TestReadStripedFileWithDecoding>Object.wait:-2 » test timed out after 300000 ... TestReadStripedFileWithMissingBlocks.testReadFileWithMissingBlocks:78->readFileWithMissingBlocks:115 » TestDFSStripedOutputStreamWithFailure.testAddBlockWhenNoSufficientParityNumOfNodes:310 » TestHFlush.testHFlushInterrupted » ClosedByInterrupt TestWriteReadStripedFile.testFileSmallerThanOneCell1:96->testOneFileUsingDFSStripedInputStream:200->testOneFileUsingDFSStripedInputStream:224 » TestWriteReadStripedFile.testFileSmallerThanOneCell2:102->testOneFileUsingDFSStripedInputStream:200->testOneFileUsingDFSStripedInputStream:224 » TestAclsEndToEnd.testCreateFileInEncryptionZone:946 » IO Filesystem closed TestRollingUpgradeRollback.testRollbackCommand:97 » IO Cannot lock storage /ho... TestRollingUpgradeRollback.testRollbackWithQJM:159 » IO Cannot remove data dir... TestRollingUpgradeRollback.testRollbackWithHAQJM:211 » IO Cannot lock storage ... TestRollingUpgradeDowngrade.testRejectNewFsImage » Unexpected exception, expe... TestRollingUpgradeDowngrade.testDowngrade » Unexpected exception, expected<ja... TestShortCircuitLocalRead.testSmallFileLocalRead:308->doTestShortCircuitRead:241->doTestShortCircuitReadImpl:286->checkFileContent:157 » IndexOutOfBounds TestShortCircuitLocalRead.testLocalReadLegacy:316->doTestShortCircuitReadLegacy:235->doTestShortCircuitReadImpl:286->checkFileContent:157 » IndexOutOfBounds TestShortCircuitLocalRead.testLocalReadFallback:327->doTestShortCircuitReadLegacy:235->doTestShortCircuitReadImpl:286->checkFileContent:157 » IndexOutOfBounds TestFileLengthOnClusterRestart.testFileLengthWithHSyncAndClusterRestartWithOutDNsRegister:42 » IO TestReplication.testBadBlockReportOnTransferMissingBlockFile:222->testBadBlockReportOnTransfer:179 » IllegalArgument Tests run: 4360, Failures: 1, Errors: 14, Skipped: 17 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS Native Client [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HttpFS [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS BookKeeper Journal [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS-NFS [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop HDFS Project 3.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-hdfs-project --- [INFO] Deleting <https://builds.apache.org/job/Hadoop-Hdfs-trunk/ws/hadoop-hdfs-project/target> [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-hdfs-project --- [INFO] Executing tasks main: [mkdir] Created dir: <https://builds.apache.org/job/Hadoop-Hdfs-trunk/ws/hadoop-hdfs-project/target/test-dir> [INFO] Executed tasks [INFO] [INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-hdfs-project --- [INFO] Skipping javadoc generation [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ hadoop-hdfs-project --- [INFO] [INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ hadoop-hdfs-project --- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop HDFS Client ......................... SUCCESS [04:00 min] [INFO] Apache Hadoop HDFS ................................ FAILURE [ 04:01 h] [INFO] Apache Hadoop HDFS Native Client .................. SKIPPED [INFO] Apache Hadoop HttpFS .............................. SKIPPED [INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED [INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [ 0.069 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 04:05 h [INFO] Finished at: 2016-04-14T22:19:57+00:00 [INFO] Final Memory: 57M/671M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-hdfs: There was a timeout or other error in the fork -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-hdfs Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results