Jenkins build is back to normal : Hadoop-Hdfs-trunk-Java8 #1277

2016-05-27 Thread Apache Jenkins Server 
See 


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1278

2016-05-27 Thread Apache Jenkins Server 
See 

Changes:

[kai.zheng] HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by 
Jiajia

[yzhang] HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun

[xgong] YARN-5089. Improve "yarn log" command-line "logFiles" option to support

--
[...truncated 5757 lines...]
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.118 sec - in 
org.apache.hadoop.hdfs.tools.TestDebugAdmin
Running org.apache.hadoop.hdfs.tools.TestDFSAdmin
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.695 sec - in 
org.apache.hadoop.hdfs.tools.TestStoragePolicyCommands
Running org.apache.hadoop.hdfs.tools.TestDelegationTokenFetcher
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.789 sec - in 
org.apache.hadoop.hdfs.tools.TestDFSZKFailoverController
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.553 sec - in 
org.apache.hadoop.hdfs.tools.TestDelegationTokenFetcher
Running org.apache.hadoop.hdfs.TestCrcCorruption
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.388 sec - in 
org.apache.hadoop.hdfs.tools.TestDFSAdmin
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.821 sec - in 
org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
Running org.apache.hadoop.hdfs.TestDFSRename
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.898 sec - in 
org.apache.hadoop.hdfs.TestDFSRename
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.447 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030
Running org.apache.hadoop.hdfs.TestDatanodeConfig
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.672 sec - in 
org.apache.hadoop.hdfs.TestCrcCorruption
Running org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.311 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.182 sec - in 
org.apache.hadoop.hdfs.TestDatanodeConfig
Running org.apache.hadoop.hdfs.TestFileAppend2
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.853 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestSetrepIncreasing
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.737 sec - in 
org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.749 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.351 sec - in 
org.apache.hadoop.hdfs.TestFileAppend2
Running org.apache.hadoop.hdfs.TestDFSClientRetries
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.443 sec - in 
org.apache.hadoop.hdfs.TestSetrepIncreasing
Running org.apache.hadoop.hdfs.TestAsyncDFS
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.292 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Running org.apache.hadoop.hdfs.TestHdfsAdmin
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.348 sec - in 
org.apache.hadoop.hdfs.TestHdfsAdmin
Running org.apache.hadoop.hdfs.TestDataTransferKeepalive
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.471 sec - in 
org.apache.hadoop.hdfs.TestAsyncDFS
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.77 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.579 sec - in 
org.apache.hadoop.hdfs.TestDataTransferKeepalive
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.008 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
Running org.apache.hadoop.hdfs.TestDatanodeDeath
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.01 sec - in 
org.apache.hadoop.hdfs.TestDatanodeDeath
Running org.apache.hadoop.hdfs.TestHDFSFileSystemContract
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 92.873 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
Running org.apache.hadoop.hdfs.TestHFlush
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 139.485 sec - 
in org.apache.hadoop.hdfs.TestDFSClientRetries
Running org.apac

Hadoop-Hdfs-trunk-Java8 - Build # 1278 - Failure

2016-05-27 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1278/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5954 lines...]
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:21 min]
[INFO] Apache Hadoop HDFS . FAILURE [59:11 min]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.181 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:03 h
[INFO] Finished at: 2016-05-27T08:32:27+00:00
[INFO] Final Memory: 114M/3739M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: ExecutionException: java.lang.RuntimeException: 
java.lang.RuntimeException: java.io.IOException: Stream Closed -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery.testDnRestartWithSavedReplicas

Error Message:

Expected: is 
 but: was 

Stack Trace:
java.lang.AssertionError: 
Expected: is 
 but: was 
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.junit.Assert.assertThat(Assert.java:865)
at org.junit.Assert.assertThat(Assert.java:832)
at 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.LazyPersistTestCase.ensureFileReplicasOnStorageType(LazyPersistTestCase.java:141)
at 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery.testDnRestartWithSavedReplicas(TestLazyPersistReplicaRecovery.java:53)

Jenkins build is back to normal : Hadoop-Hdfs-trunk-Java8 #1279

2016-05-27 Thread Apache Jenkins Server 
See 


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-10467) Router-based HDFS federation

2016-05-27 Thread Inigo Goiri (JIRA) 
Inigo Goiri created HDFS-10467:
--

 Summary: Router-based HDFS federation
 Key: HDFS-10467
 URL: https://issues.apache.org/jira/browse/HDFS-10467
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: fs
Affects Versions: 2.7.2
Reporter: Inigo Goiri


Add a Router to provide a federated view of multiple HDFS clusters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-10468) HDFS read ends up ignoring InterruptedException

2016-05-27 Thread Siddharth Seth (JIRA) 
Siddharth Seth created HDFS-10468:
-

 Summary: HDFS read ends up ignoring InterruptedException
 Key: HDFS-10468
 URL: https://issues.apache.org/jira/browse/HDFS-10468
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Siddharth Seth


If an interrupt comes in during an HDFS read - it looks like HDFS ends up 
ignoring it (handling it), and retries the read after an interval.

An interrupt should result in the read being cancelled, with an 
InterruptedException being thrown.

Similarly - if an HDFS op is started with the interrupt status on the thread 
set, an InterruptedException should be thrown.

cc [~jingzhao]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1280 - Failure

2016-05-27 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1280/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 6151 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:00 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:09 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.091 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:13 h
[INFO] Finished at: 2016-05-27T18:58:40+00:00
[INFO] Final Memory: 95M/3951M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/source/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
3 tests failed.
FAILED:  org.apache.hadoop.hdfs.TestEncryptionZones.testStartFileRetry

Error Message:
test timed out after 12 milliseconds

Stack Trace:
java.lang.Exception: test timed out after 12 milliseconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at 
org.apache.hadoop.hdfs.TestEncr

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1280

2016-05-27 Thread Apache Jenkins Server 
See 

Changes:

[yzhang] HDFS-10276. HDFS should not expose path info that user has no 
permission

--
[...truncated 5954 lines...]

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
Running org.apache.hadoop.hdfs.TestDFSRemove
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.544 sec - in 
org.apache.hadoop.hdfs.TestDFSInputStream
Running org.apache.hadoop.hdfs.TestFileAppend4
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.153 sec - in 
org.apache.hadoop.hdfs.TestDFSRemove
Running org.apache.hadoop.hdfs.TestParallelRead
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.44 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 99.352 sec - 
in org.apache.hadoop.hdfs.TestReconstructStripedFile
Running org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.904 sec - in 
org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSAddressConfig
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.361 sec - in 
org.apache.hadoop.hdfs.TestFileAppend4
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.951 sec - in 
org.apache.hadoop.hdfs.TestDFSAddressConfig
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.426 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.287 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Running org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.295 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.082 sec - in 
org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.492 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.157 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.973 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.795 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.737 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.718 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.772 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 68.08 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.268 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.66 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.688 sec - in 
org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Running org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.176 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderFactory
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.293 sec - in 
org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.432 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Running org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
Running org.apac

Re: Another thought on client-side support of HDFS federation

2016-05-27 Thread Chris Nauroth 
Hello Tianyi HE,

I noticed that a similar design for a federation proxying model has just
been proposed on Apache JIRA HDFS-10467.  You might want to join the
conversation there.

https://issues.apache.org/jira/browse/HDFS-10467


--Chris Nauroth




On 5/2/16, 10:32 AM, "Colin McCabe"  wrote:

>Hi Tianyi HE,
>
>Thanks for sharing this!  This reminds me of the httpfs daemon.  This
>daemon basically sits in front of an HDFS cluster and accepts requests,
>which it serves by forwarding them to the underlying HDFS instance.
>There is some documentation about it here:
>https://hadoop.apache.org/docs/stable/hadoop-hdfs-httpfs/index.html
>
>Since httpfs uses an org.apache.hadoop.fs.FileSystem instance, it seems
>like you could plug in the apache.hadoop.fs.viewfs.ViewFileSystem class
>and be up and running with federation.  I haven't tried this, but I
>would expect that it would work, unless there are bugs in ViewFS itself.
>
>The big advantage of httpfs is that it provides a webhdfs-style REST
>interface.  As you said, this kind of interface makes it simple to use
>any language with REST bindings, without worrying about using a thick
>client.
>
>The big disadvantage of httpfs is that you must move both metadata and
>data operations through the httpfs daemon.  This could become a
>performance bottleneck.  It seems like you are concerned about this
>bottleneck.
>
>We also have webhdfs.  Unlike httpfs, webhdfs doesn't require all the
>data to move through its daemon.  With webhdfs, the client talks to
>DataNodes directly.
>
>I wonder if extending httpfs or webhdfs would be a better approach than
>starting from scratch.  There is a maintenance burden for adding new
>services and daemons.  This was our motivation for removing hftp, for
>example.  It's certainly something to think about.
>
>best,
>Colin
>
>
>On Thu, Apr 28, 2016, at 17:55, 何天一 wrote:
>> Hey guys,
>> 
>> My associates have investigated HDFS federation recently, which, turns
>> out
>> to be a quite good solution for improving scalability on
>> NameNode/DataNode
>> side.
>> 
>> However, we encountered some problem on client-side. Since:
>> A) For historical reason, we use clients in multiple languages to access
>> HDFS, (i.e. python-snakebite, or perhaps libhdfs++). So we either
>> implement
>> multiple versions of ViewFS or we give up the consistency view (which
>>can
>> be confusing to user).
>> B) We have hadoop client configuration deployed on client nodes, which
>>we
>> do not have control over . Also, releasing new configuration could be a
>> real heavy operation because it needs to be pushed to several thousand
>>of
>> nodes, as well as maintaining consistency (say a node is down throughout
>> the operation, then come back online. it could still possess a stale
>> version of configuration).
>> 
>> So we intended to explore another solution to these problems, and came
>>up
>> with a proxy model.
>> That is, build a RPC proxy in front of NameNodes.
>> All clients talk to proxy when they need to consult NameNode, then proxy
>> decide which NameNode should the request go to according to mount table.
>> This solved our problem. All clients are seamlessly upgraded with
>> federation support.
>> We open sourced the proxy recently: https://github.com/bytedance/nnproxy
>> (BTW, all kinds of feedbacks are welcomed)
>> 
>> But there are still a few issues. For example, several modifications
>> needs
>> to be done inside hadoop ipc to support rpc forwarding. We released
>>patch
>> according to which with nnproxy project (
>> https://github.com/bytedance/nnproxy/tree/master/hadoop-patches). But it
>> could be better to have these merged to apache trunk. Does someone think
>> it's worth?
>> 
>> 
>> -- 
>> Cheers,
>> Tianyi HE
>> (+86) 185 0042 4096
>
>-
>To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1281

2016-05-27 Thread Apache Jenkins Server 
See 

Changes:

[Arun Suresh] YARN-5127. Expose ExecutionType in Container api record. (Hitesh 
Sharma

[cnauroth] HADOOP-13193. Upgrade to Apache Yetus 0.3.0. Contributed by Kengo 
Seki.

[cnauroth] HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by

--
[...truncated 8864 lines...]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.406 sec - in 
org.apache.hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.304 sec - 
in org.apache.hadoop.hdfs.TestEncryptionZones
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 66.226 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.TestSnapshotCommands
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.038 sec - in 
org.apache.hadoop.hdfs.TestSnapshotCommands
Running org.apache.hadoop.hdfs.TestDataTransferProtocol
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.758 sec - in 
org.apache.hadoop.hdfs.TestDataTransferProtocol
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.267 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Running org.apache.hadoop.hdfs.TestFileCreation
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.128 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Running org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.176 sec - in 
org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Tests run: 44, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 183.987 sec - 
in org.apache.hadoop.hdfs.TestHDFSFileSystemContract
Running org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.127 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Running org.apache.hadoop.hdfs.TestDecommission
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.681 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Running org.apache.hadoop.hdfs.TestDFSUtil
Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.163 sec - in 
org.apache.hadoop.hdfs.TestDFSUtil
Running org.apache.hadoop.hdfs.TestGetBlocks
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.408 sec - 
in org.apache.hadoop.hdfs.TestFileCreation
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.212 sec - in 
org.apache.hadoop.hdfs.TestGetBlocks
Running org.apache.hadoop.hdfs.TestMultiThreadedHflush
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.869 sec - in 
org.apache.hadoop.hdfs.TestMultiThreadedHflush
Running org.apache.hadoop.hdfs.util.TestCyclicIteration
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.089 sec - in 
org.apache.hadoop.hdfs.util.TestCyclicIteration
Running org.apache.hadoop.hdfs.util.TestBestEffortLongFile
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.301 sec - in 
org.apache.hadoop.hdfs.util.TestBestEffortLongFile
Running org.apache.hadoop.hdfs.util.TestDiff
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.703 sec - in 
org.apache.hadoop.hdfs.util.TestDiff
Running org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.004 sec - in 
org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Running org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.342 sec - in 
org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Running org.apache.hadoop.hdfs.util.TestXMLUtils
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec - in 
org.apache.hadoop.hdfs.util.TestXMLUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.403 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Running org.apache.hadoop.hdfs.util.TestMD5FileUtils
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.547 sec - in 
org.apache.hadoop.hdfs.util.TestMD5FileUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Running org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.394 sec - in 
org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Run

Hadoop-Hdfs-trunk-Java8 - Build # 1281 - Still Failing

2016-05-27 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1281/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 9061 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [03:56 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:08 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.091 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:12 h
[INFO] Finished at: 2016-05-27T22:58:04+00:00
[INFO] Final Memory: 95M/3815M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/source/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testBalancerWithKeytabs

Error Message:
test timed out after 30 milliseconds

Stack Trace:
java.lang.Exception: test timed out after 30 milliseconds
at java.lang.Thread.sleep(Native Method)
at 
org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:705)
at 
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
at 
org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)


FAILED:  
org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot.testParentDirWithUCFileDeleteWithSnapShot

Error Message:
Timed out waiting for Mini HDFS Cluster to start

Stack Trace:
java.io.IOException: Timed out waiting for Mini HDFS Cluster to start

[jira] [Created] (HDFS-10469) Add number of active xceivers to datanode metrics

2016-05-27 Thread Hanisha Koneru (JIRA) 
Hanisha Koneru created HDFS-10469:
-

 Summary: Add number of active xceivers to datanode metrics
 Key: HDFS-10469
 URL: https://issues.apache.org/jira/browse/HDFS-10469
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: datanode
Affects Versions: 3.0.0-alpha1
Reporter: Hanisha Koneru
Assignee: Hanisha Koneru


Number of active xceivers is exposed via jmx, but not in Datanode metrics. We 
should add it to datanode metrics for monitoring the load on Datanodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1282

2016-05-27 Thread Apache Jenkins Server 
See 

Changes:

[Arun Suresh] YARN-5117. QueuingContainerManager does not start GUARANTEED 
Container

--
[...truncated 5286 lines...]
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.236 sec - in 
org.apache.hadoop.hdfs.util.TestBestEffortLongFile
Running org.apache.hadoop.hdfs.util.TestDiff
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.462 sec - in 
org.apache.hadoop.hdfs.util.TestDiff
Running org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.928 sec - in 
org.apache.hadoop.hdfs.TestGetBlocks
Running org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.33 sec - in 
org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Running org.apache.hadoop.hdfs.util.TestXMLUtils
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec - in 
org.apache.hadoop.hdfs.util.TestXMLUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.192 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Running org.apache.hadoop.hdfs.util.TestMD5FileUtils
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.145 sec - in 
org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.446 sec - in 
org.apache.hadoop.hdfs.util.TestMD5FileUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Running org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.211 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Running org.apache.hadoop.hdfs.TestLease
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.373 sec - in 
org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Running org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.333 sec - in 
org.apache.hadoop.hdfs.TestLease
Running org.apache.hadoop.hdfs.TestHFlush
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.441 sec - in 
org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
Running org.apache.hadoop.hdfs.TestErasureCodingPolicies
Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 99.675 sec - 
in org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
Running org.apache.hadoop.hdfs.TestHDFSPolicyProvider
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.229 sec - in 
org.apache.hadoop.hdfs.TestHDFSPolicyProvider
Running org.apache.hadoop.hdfs.TestHdfsAdmin
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.053 sec - in 
org.apache.hadoop.hdfs.TestHdfsAdmin
Running org.apache.hadoop.hdfs.TestDistributedFileSystem
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.707 sec - 
in org.apache.hadoop.hdfs.TestErasureCodingPolicies
Running org.apache.hadoop.hdfs.TestRollingUpgradeRollback
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.737 sec - in 
org.apache.hadoop.hdfs.TestRollingUpgradeRollback
Running org.apache.hadoop.hdfs.TestRollingUpgrade
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.302 sec - 
in org.apache.hadoop.hdfs.TestHFlush
Running org.apache.hadoop.hdfs.TestDatanodeDeath
Tests run: 12, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 44.527 sec <<< 
FAILURE! - in org.apache.hadoop.hdfs.TestRollingUpgrade
testCheckpointWithMultipleNN(org.apache.hadoop.hdfs.TestRollingUpgrade)  Time 
elapsed: 4.647 sec  <<< FAILURE!
java.lang.AssertionError: Test resulted in an unexpected exit
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1913)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1900)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1893)
at 
org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster.shutdown(MiniQJMHACluster.java:168)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testCheckpoint(TestRollingUpgrade.java:604)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testCheckpointWithMultipleNN(TestRollingUpgrade.java:568)

testDFSAdminRollingUpgradeCommands(org.apache.hadoop.hdfs.TestRollingUpgrade)  
Time elapsed: 0.888 sec  <<< FAILURE!
java.lang.AssertionError: expected null, but 
was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotNull(Assert.java:664)
at org.junit.Assert.assertNull(Assert.java:646)
at org.junit.Assert.assertNull(Assert.java:656)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.checkMxBeanIsNull(TestRollingUpgrade.java:295)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testDFSAdminRollingUpgradeCommands(TestRollingUpgrade.java:102)

testRollback(org.apache.ha

Hadoop-Hdfs-trunk-Java8 - Build # 1282 - Still Failing

2016-05-27 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1282/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5483 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:16 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:06 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.092 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:11 h
[INFO] Finished at: 2016-05-28T02:56:31+00:00
[INFO] Final Memory: 95M/3847M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/source/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
5 tests failed.
FAILED:  org.apache.hadoop.hdfs.TestRollingUpgrade.testCheckpointWithMultipleNN

Error Message:
Test resulted in an unexpected exit

Stack Trace:
java.lang.AssertionError: Test resulted in an unexpected exit
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1913)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1900)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1893)
at 
org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster.shutdown(MiniQJMHACluster.java:168)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testCheckpoint(TestRollingUpgrade.java:604)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testCheckpointWithMultipleNN(TestRollingUpgrade.java:568)


FAILED:  
org.apache.hadoop.hdfs.TestRollingUpgrade.testDFSAdminR

Hadoop-Hdfs-trunk-Java8 - Build # 1283 - Still Failing

2016-05-27 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1283/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5987 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [03:56 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:06 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.081 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:10 h
[INFO] Finished at: 2016-05-28T04:08:53+00:00
[INFO] Final Memory: 92M/3641M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/source/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache.testPageRounder

Error Message:
Timed out waiting for condition. Thread diagnostics:
Timestamp: 2016-05-28 03:29:08,860

"IPC Server handler 8 on 43729" daemon prio=5 tid=60 timed_waiting
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at 
org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:218)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2387)
"IPC Server listener on 43729" daemon pri

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1283

2016-05-27 Thread Apache Jenkins Server 
See 

Changes:

[xyao] HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler.

--
[...truncated 5790 lines...]
Running org.apache.hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.72 sec - in 
org.apache.hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.348 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.569 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestSnapshotCommands
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.251 sec - in 
org.apache.hadoop.hdfs.TestSnapshotCommands
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.933 sec - 
in org.apache.hadoop.hdfs.TestEncryptionZones
Running org.apache.hadoop.hdfs.TestDataTransferProtocol
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.343 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
Running org.apache.hadoop.hdfs.TestFileCreation
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.227 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.115 sec - in 
org.apache.hadoop.hdfs.TestDataTransferProtocol
Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.341 sec - in 
org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Running org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.945 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Running org.apache.hadoop.hdfs.TestDecommission
Tests run: 44, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 175.978 sec - 
in org.apache.hadoop.hdfs.TestHDFSFileSystemContract
Running org.apache.hadoop.hdfs.TestDFSUtil
Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.909 sec - in 
org.apache.hadoop.hdfs.TestDFSUtil
Running org.apache.hadoop.hdfs.TestGetBlocks
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.823 sec - 
in org.apache.hadoop.hdfs.TestFileCreation
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.35 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
Running org.apache.hadoop.hdfs.TestMultiThreadedHflush
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.077 sec - in 
org.apache.hadoop.hdfs.TestMultiThreadedHflush
Running org.apache.hadoop.hdfs.util.TestCyclicIteration
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.084 sec - in 
org.apache.hadoop.hdfs.util.TestCyclicIteration
Running org.apache.hadoop.hdfs.util.TestBestEffortLongFile
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.271 sec - in 
org.apache.hadoop.hdfs.util.TestBestEffortLongFile
Running org.apache.hadoop.hdfs.util.TestDiff
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.694 sec - in 
org.apache.hadoop.hdfs.util.TestDiff
Running org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.354 sec - in 
org.apache.hadoop.hdfs.util.TestStripedBlockUtil
Running org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.338 sec - in 
org.apache.hadoop.hdfs.util.TestCombinedHostsFileReader
Running org.apache.hadoop.hdfs.util.TestXMLUtils
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec - in 
org.apache.hadoop.hdfs.util.TestXMLUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.2 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightHashSet
Running org.apache.hadoop.hdfs.util.TestMD5FileUtils
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.416 sec - in 
org.apache.hadoop.hdfs.util.TestMD5FileUtils
Running org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.202 sec - in 
org.apache.hadoop.hdfs.util.TestLightWeightLinkedSet
Running org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.378 sec - in 
org.apache.hadoop.hdfs.util.TestAtomicFileOutputStream
Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 103.646 sec - 
in org.apache.had