Hadoop-Hdfs-trunk - Build # 1372 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1372/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 14301 lines...]
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints.testStandbyExceptionThrownDuringCheckpoint(TestStandbyCheckpoints.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at 
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)

Running org.apache.hadoop.contrib.bkjournal.TestCurrentInprogress
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.679 sec
Running org.apache.hadoop.contrib.bkjournal.TestBookKeeperConfiguration
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.061 sec
Running org.apache.hadoop.contrib.bkjournal.TestBookKeeperJournalManager
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.132 sec

Results :

Failed tests:   
testStandbyExceptionThrownDuringCheckpoint(org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints):
 SBN should have still been checkpointing.

Tests run: 32, Failures: 1, Errors: 0, Skipped: 0

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  SUCCESS 
[1:28:42.696s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [2:27.200s]
[INFO] Apache Hadoop HDFS BookKeeper Journal . FAILURE [57.696s]
[INFO] Apache Hadoop HDFS Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:32:08.359s
[INFO] Finished at: Sun Apr 14 13:06:01 UTC 2013
[INFO] Final Memory: 51M/643M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on 
project hadoop-hdfs-bkjournal: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs-bkjournal
Build step 'Execute shell' marked build as failure
Archiving artifacts
Updating HADOOP-9473
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1372

[Apache Jenkins Server]

See 

Changes:

[suresh] HADOOP-9473. Add change description missed in previous commit 1467694

[suresh] HADOOP-9473. Typo in FileUtil copy() method. Contributed by Glen Mazza.

--
[...truncated 14108 lines...]
Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Re: MapReduce Integration tests for test FileSystems

[Steve Loughran]

On 13 April 2013 21:46, Jay Vyas  wrote:

> MapReduce has some very demanding file operations at at job submission
> time, and is an important integration test for any hadoop compliant
> FileSystem.
>
> How in the hadoop HDFS source tree are mapreduce integration tests
> implemented (also I'm interested in how S3, Swift, and other Hadoop
> filesystem implementations) implement integration tests... if any clues.  I
> haven't seen much in this way in the source code.
>


There are some functional tests for S3 that run by hand; when the swift
support goes in that will be skipped unless there is the file
test/resources/auth-keys.xml containing the login details.

We'd like to add more here in Bigtop, which is downstream enough you can
include stuff like Pig, Hive and HBase tests, using different filesystems
as source, destination and intermediate files. They should also allow the
option of creating big files, many files in a single dir and deeper
directories: scale problems.




> To distill my quesiton: how are new HDFS patches tested against the
> MapReduce job flow --- ? --- are there some standalone vanilla MapReduce
> jobs (Sorting, WordCount, etc..., that run as part of the HDFS build or is
> there an HDFS-Mapreduce integration repository ?)
>
>
The -common, HDFS and -mapred source trees are in a combined repo so this
is implicit, you just go "mvn clean test" at the root. Or have Jenkins do
it and send email when you break things

steve

[jira] [Created] (HDFS-4694) The SecondaryNameNode may change the inode id of root while doing checkpoint

[Jing Zhao (JIRA)]

Jing Zhao created HDFS-4694:
---

 Summary: The SecondaryNameNode may change the inode id of root 
while doing checkpoint
 Key: HDFS-4694
 URL: https://issues.apache.org/jira/browse/HDFS-4694
 Project: Hadoop HDFS
  Issue Type: Sub-task
Affects Versions: 3.0.0
Reporter: Jing Zhao
Assignee: Jing Zhao


SNN will call FSNamesystem#clear while merging the fsimage to its local 
FSNamesystem, and the FSDirectory will be reset in this process. However, in 
FSDirectory#reset, a new inode id will be allocated and assigned to the new 
root.

To fix the problem, we can either pass in an inode id in 
FSDirectory#createRoot, or update the root id in FSImageFormat#updateRootAttr.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-4695) TestEditLog leaks open file handles between tests

[Ivan Mitic (JIRA)]

Ivan Mitic created HDFS-4695:


 Summary: TestEditLog leaks open file handles between tests
 Key: HDFS-4695
 URL: https://issues.apache.org/jira/browse/HDFS-4695
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic


The test leaks open file handles causing subsequent test cases to fail on 
Windows (common cross-platform issue we've seen multiple times so far).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira