Hadoop-Hdfs-trunk - Build # 910 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/910/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 12078 lines...]
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is true
[INFO] ** FindBugsMojo executeFindbugs ***
[INFO] Temp File is 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/findbugsTemp.xml
[INFO] Fork Value is true
[INFO] xmlOutput is false
[INFO] 
[INFO] 
[INFO] Building Apache Hadoop HDFS Project 0.24.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-hdfs-project 
---
[INFO] Deleting 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/target
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-hdfs-project 
---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/target/test-dir
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-javadoc-plugin:2.7:jar (module-javadocs) @ hadoop-hdfs-project 
---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.6:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is false
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  SUCCESS [4:59.826s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [30.959s]
[INFO] Apache Hadoop HDFS BookKeeper Journal . SUCCESS [10.746s]
[INFO] Apache Hadoop HDFS Project  SUCCESS [0.036s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 5:41.990s
[INFO] Finished at: Fri Dec 30 11:40:36 UTC 2011
[INFO] Final Memory: 84M/747M
[INFO] 
+ /home/jenkins/tools/maven/latest/bin/mvn test 
-Dmaven.test.failure.ignore=true -Pclover 
-DcloverLicenseLocation=/home/jenkins/tools/clover/latest/lib/clover.license
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Publishing Javadoc
Recording fingerprints
Updating MAPREDUCE-3608
Updating HDFS-2729
Updating MAPREDUCE-3522
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
4 tests failed.
REGRESSION:  
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes.testBalancer

Error Message:
Cannot lock storage 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/name1.
 The directory is already locked.

Stack Trace:
java.io.IOException: Cannot lock storage 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/name1.
 The directory is already locked.
at 
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:586)
at 
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:435)
at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:253)
at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:169)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:385)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:328)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:299)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:333)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:459)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:451)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNod

Build failed in Jenkins: Hadoop-Hdfs-trunk #910

[Apache Jenkins Server]

See 

Changes:

[acmurthy] MAPREDUCE-3608. Fixed compile issue with MAPREDUCE-3522. Contributed 
by Mahadev Konar.

[acmurthy] MAPREDUCE-3522. Ensure queues inherit ACLs from parent if they 
aren't explicitly specified. Contributed by Jonathan Eagles.

[harsh] HDFS-2729. Update BlockManager's comments regarding the invalid block 
set (harsh)

--
[...truncated 11885 lines...]
Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

[jira] [Created] (HDFS-2735) HA: add tests for multiple shared edits dirs

[Eli Collins (Created) (JIRA)]

HA: add tests for multiple shared edits dirs


 Key: HDFS-2735
 URL: https://issues.apache.org/jira/browse/HDFS-2735
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, test
Affects Versions: HA branch (HDFS-1623)
Reporter: Eli Collins


You can configure and run with multiple shared edits dirs but we don't have any 
test coverage for them. In particular, we should cover the behavior of the edit 
log tailer with multiple dirs, and failure scenarios (eg can we tolerate a 
single shared dir failure if we have two shared dirs).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-2736) HA: support separate SBN and 2NN?

[Eli Collins (Created) (JIRA)]

HA: support separate SBN and 2NN?
-

 Key: HDFS-2736
 URL: https://issues.apache.org/jira/browse/HDFS-2736
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha
Affects Versions: HA branch (HDFS-1623)
Reporter: Eli Collins


HDFS-2291 adds support for making the SBN capable of checkpointing, seems like 
we may also need to support the 2NN checkpointing as well. Eg if we fail over 
to the SBN does it continue to checkpoint? If not the log grows unbounded until 
the old primary comes back, if so does that create performance problems since 
the primary wasn't previously checkpointing?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-2737) HA: Automatically trigger log rolls periodically on the active NN

[Todd Lipcon (Created) (JIRA)]

HA: Automatically trigger log rolls periodically on the active NN
-

 Key: HDFS-2737
 URL: https://issues.apache.org/jira/browse/HDFS-2737
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon


Currently, the edit log tailing process can only read finalized log segments. 
So, if the active NN is not rolling its logs periodically, the SBN will lag a 
lot. This also causes many datanode messages to be queued up in the 
PendingDatanodeMessage structure.

To combat this, the active NN needs to roll its logs periodically -- perhaps 
based on a time threshold, or perhaps based on a number of transactions. I'm 
not sure yet whether it's better to have the NN roll on its own or to have the 
SBN ask the active NN to roll its logs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-2736) HA: support separate SBN and 2NN?

[Eli Collins (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eli Collins resolved HDFS-2736.
---

Resolution: Won't Fix

I was thinking about the case where the former active remains dead for some 
time. But this case is problematic for a number of other reasons (eg with just 
a single host we can't failback if necessary) so I think its reasonable to 
require users start another SBN instead also deploying a 2NN. Closing as won't 
fix, can re-open if others disagree.

> HA: support separate SBN and 2NN?
> -
>
> Key: HDFS-2736
> URL: https://issues.apache.org/jira/browse/HDFS-2736
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ha
>Affects Versions: HA branch (HDFS-1623)
>Reporter: Eli Collins
>
> HDFS-2291 adds support for making the SBN capable of checkpointing, seems 
> like we may also need to support the 2NN checkpointing as well. Eg if we fail 
> over to the SBN does it continue to checkpoint? If not the log grows 
> unbounded until the old primary comes back, if so does that create 
> performance problems since the primary wasn't previously checkpointing?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-2738) FSEditLog.selectinputStreams is reading through in-progress streams even when non-in-progress are requested

[Todd Lipcon (Created) (JIRA)]

FSEditLog.selectinputStreams is reading through in-progress streams even when 
non-in-progress are requested
---

 Key: HDFS-2738
 URL: https://issues.apache.org/jira/browse/HDFS-2738
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical


The new code in HDFS-1580 is causing an issue with selectInputStreams in the HA 
context. When the active is writing to the shared edits, selectInputStreams is 
called on the standby. This ends up calling {{journalSet.getInputStream}} but 
doesn't pass the {{inProgressOk=false}} flag. So, {{getInputStream}} ends up 
reading and validating the in-progress stream unnecessarily. Since the 
validation results are no longer properly cached, {{findMaxTransaction}} also 
re-validates the in-progress stream, and then breaks the corruption check in 
this code. The end result is a lot of errors like:

2011-12-30 16:45:02,521 ERROR namenode.FileJournalManager 
(FileJournalManager.java:getNumberOfTransactions(266)) - Gap in transactions, 
max txnid is 579, 0 txns from 578
2011-12-30 16:45:02,521 INFO  ha.EditLogTailer (EditLogTailer.java:run(163)) - 
Got error, will try again.
java.io.IOException: No non-corrupt logs for txid 578
at 
org.apache.hadoop.hdfs.server.namenode.JournalSet.getInputStream(JournalSet.java:229)
at 
org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1081)
at 
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:115)
at 
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.access$0(EditLogTailer.java:100)
at 
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:154)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-2716) HA: Configuration needs to allow different dfs.http.addresses for each HA NN

[Todd Lipcon (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon resolved HDFS-2716.
---

   Resolution: Fixed
Fix Version/s: HA branch (HDFS-1623)
 Hadoop Flags: Reviewed

> HA: Configuration needs to allow different dfs.http.addresses for each HA NN
> 
>
> Key: HDFS-2716
> URL: https://issues.apache.org/jira/browse/HDFS-2716
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ha, name-node
>Affects Versions: HA branch (HDFS-1623)
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
> Fix For: HA branch (HDFS-1623)
>
> Attachments: hdfs-2716.txt, hdfs-2716.txt
>
>
> Earlier on the HA branch we expanded the configuration so that different IPC 
> addresses can be specified for each of the HA NNs in a cluster. But we didn't 
> do this for the HTTP address. This has proved problematic while working on 
> HDFS-2291 (checkpointing in HA).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-2739) SecondaryNameNode doesn't start up

[Sho Shimauchi (Created) (JIRA)]

SecondaryNameNode doesn't start up
--

 Key: HDFS-2739
 URL: https://issues.apache.org/jira/browse/HDFS-2739
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Sho Shimauchi
Priority: Critical


Built a 0.24-SNAPSHOT tar from today, used a general config, started NN/DN, but 
SNN won't come up with following error:

{code}
11/12/31 12:13:14 ERROR namenode.SecondaryNameNode: Throwable Exception in 
doCheckpoint
java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:154)
at 
org.apache.hadoop.ipc.WritableRpcEngine$Invocation.(WritableRpcEngine.java:112)
at 
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:226)
at $Proxy9.getTransationId(Unknown Source)
at 
org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.getTransactionID(NamenodeProtocolTranslatorPB.java:185)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.countUncheckpointedTxns(SecondaryNameNode.java:625)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.shouldCheckpointBasedOnCount(SecondaryNameNode.java:633)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:386)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:356)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NoSuchFieldException: versionID
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:150)
... 9 more
java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:154)
at 
org.apache.hadoop.ipc.WritableRpcEngine$Invocation.(WritableRpcEngine.java:112)
at 
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:226)
at $Proxy9.getTransationId(Unknown Source)
at 
org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.getTransactionID(NamenodeProtocolTranslatorPB.java:185)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.countUncheckpointedTxns(SecondaryNameNode.java:625)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.shouldCheckpointBasedOnCount(SecondaryNameNode.java:633)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:386)
at 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:356)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NoSuchFieldException: versionID
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:150)
... 9 more
11/12/31 12:13:14 INFO namenode.SecondaryNameNode: SHUTDOWN_MSG: 
/
SHUTDOWN_MSG: Shutting down SecondaryNameNode at sho-mba.local/192.168.11.2
/
{code}

full error log: http://pastebin.com/mSaVbS34




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira