Hadoop-Hdfs-trunk - Build # 1317 - Still Failing

2013-02-15 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1317/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 13693 lines...]
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at 
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Running org.apache.hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.147 sec
Running org.apache.hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.692 sec

Results :

Tests in error: 
  
testOperation[7](org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem)
  
testOperationDoAs[7](org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem)

Tests run: 283, Failures: 0, Errors: 2, Skipped: 0

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  SUCCESS 
[1:26:45.675s]
[INFO] Apache Hadoop HttpFS .. FAILURE [1:44.877s]
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:28:32.252s
[INFO] Finished at: Fri Feb 15 13:02:03 UTC 2013
[INFO] Final Memory: 42M/629M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on 
project hadoop-hdfs-httpfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs-httpfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Updating HADOOP-9218
Updating HADOOP-9154
Updating HADOOP-9304
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1317

2013-02-15 Thread Apache Jenkins Server 
See 

Changes:

[sradia] HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia)

[tucu] HADOOP-9304. remove addition of avro genreated-sources dirs to build. 
(tucu)

[tomwhite] HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value 
classes to the map. Contributed by Karthik Kambatla.

--
[...truncated 13500 lines...]
main:
[mkdir] Created dir: 

[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ 
hadoop-hdfs-httpfs ---
[INFO] Using default encoding to copy filtered resources.
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
hadoop-hdfs-httpfs ---
[INFO] Compiling 56 source files to 

[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-web-xmls) @ hadoop-hdfs-httpfs 
---
[INFO] Executing tasks

main:
[mkdir] Created dir: 

 [copy] Copying 1 file to 

[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ 
hadoop-hdfs-httpfs ---
[INFO] Using default encoding to copy filtered resources.
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
hadoop-hdfs-httpfs ---
[INFO] Compiling 46 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ 
hadoop-hdfs-httpfs ---
[INFO] Surefire report directory: 


---
 T E S T S
---

---
 T E S T S
---
Running org.apache.hadoop.test.TestDirHelper
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec
Running org.apache.hadoop.test.TestJettyHelper
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec
Running org.apache.hadoop.test.TestHdfsHelper
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061 sec
Running org.apache.hadoop.test.TestHTestCase
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.198 sec
Running org.apache.hadoop.test.TestExceptionHelper
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec
Running org.apache.hadoop.test.TestHFSTestCase
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.824 sec
Running org.apache.hadoop.lib.service.instrumentation.TestInstrumentationService
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.671 sec
Running org.apache.hadoop.lib.service.scheduler.TestSchedulerService
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.21 sec
Running org.apache.hadoop.lib.service.security.TestProxyUserService
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 sec
Running org.apache.hadoop.lib.service.security.TestDelegationTokenManagerService
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 sec
Running org.apache.hadoop.lib.service.security.TestGroupsService
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.213 sec
Running org.apache.hadoop.lib.service.hadoop.TestFileSystemAccessService
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.084 sec
Running org.apache.hadoop.lib.server.TestServerConstructor
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.121 sec
Running org.apache.hadoop.lib.server.TestServer
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.436 sec
Running org.apache.hadoop.lib.server.TestBaseService
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.268 sec
Running org.apache.hadoop.lib.lang.TestRunnableCallable
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 sec
Running org.apache.hadoop.lib.lang.TestXException
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 sec
Running org.apache.hadoop.lib.wsrs.TestParam
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec
Running org.apache.hadoop.lib.wsrs.TestInputStreamEntity
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.086 sec
Running org.apache.hadoop.lib.wsrs.TestJSONProvider
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0,

Planning for hadoop-2.0.4-beta

2013-02-15 Thread Arun C Murthy 
Folks,

 Apropos to our previous discussions, I'd appreciate if folks could spend time 
over the next month or so reviewing APIs for future compatibility, bugs etc. so 
that we can lock them down - particularly in the protobuf land. I know, this is 
easier said that done - but a focussed effort should help nevertheless.

 I volunteer to do this for YARN/MapReduce (of course, any help is very much 
appreciated and I see Vinod has opened YARN-386), and would appreciate some 
folks stepping up for HDFS (Suresh? Todd?).

thanks,
Arun


--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

[jira] [Created] (HDFS-4506) HDFS short circuit fails non-transparently when user does not have unix permissions

2013-02-15 Thread Enis Soztutar (JIRA) 
Enis Soztutar created HDFS-4506:
---

 Summary: HDFS short circuit fails non-transparently when user does 
not have unix permissions
 Key: HDFS-4506
 URL: https://issues.apache.org/jira/browse/HDFS-4506
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs-client
Affects Versions: 1.1.1
Reporter: Enis Soztutar


We found a case, where if the short circuit user name is configured correctly, 
but the user does not have enough permissions in unix, DFS operations fails 
with IOException, rather than silently failing over through datanode. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-4476) HDFS-347: style cleanups

2013-02-15 Thread Aaron T. Myers (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers resolved HDFS-4476.
--

  Resolution: Fixed
Hadoop Flags: Reviewed

I've just committed this to the HDFS-347 branch. Thanks a lot for the 
contribution, Colin.

> HDFS-347: style cleanups
> 
>
> Key: HDFS-4476
> URL: https://issues.apache.org/jira/browse/HDFS-4476
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode, hdfs-client, performance
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-4476.001.patch, HDFS-4476.002.patch
>
>
> Clean up some code style issues in HDFS-347.
> DomainSocket.java
>   do not use AtomicInteger for status, add a new class
>   rename fdRef(), fdUnref(boolean), jfds, jbuf, SND_BUF_SIZE, etc.
>   do not override finalize().
>   remove some dead code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-3769) standby namenode become active fails because starting log segment fail on shared storage

2013-02-15 Thread liaowenrui (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liaowenrui resolved HDFS-3769.
--

Resolution: Fixed

this issue is resolved in bk

> standby namenode become active fails because starting log segment fail on 
> shared storage
> 
>
> Key: HDFS-3769
> URL: https://issues.apache.org/jira/browse/HDFS-3769
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ha
>Affects Versions: 2.0.0-alpha
> Environment: 3 datanode:158.1.132.18,158.1.132.19,160.161.0.143
> 2 namenode:158.1.131.18,158.1.132.19
> 3 zk:158.1.132.18,158.1.132.19,160.161.0.143
> 3 bookkeeper:158.1.132.18,158.1.132.19,160.161.0.143
> ensemble-size:2,quorum-size:2
>Reporter: liaowenrui
>Priority: Critical
> Fix For: 3.0.0, 2.0.4-beta
>
>
> 2012-08-06 15:09:46,264 ERROR 
> org.apache.hadoop.contrib.bkjournal.utils.RetryableZookeeper: Node 
> /ledgers/available already exists and this is not a retry
> 2012-08-06 15:09:46,264 INFO 
> org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager: Successfully 
> created bookie available path : /ledgers/available
> 2012-08-06 15:09:46,273 INFO 
> org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Recovering 
> unfinalized segments in 
> /opt/namenodeHa/hadoop-2.0.1/hadoop-root/dfs/name/current
> 2012-08-06 15:09:46,277 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Catching up to latest 
> edits from old active before taking over writer role in edits logs.
> 2012-08-06 15:09:46,363 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Reprocessing replication 
> and invalidation queues...
> 2012-08-06 15:09:46,363 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Marking all 
> datandoes as stale
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Total number of 
> blocks= 239
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of invalid 
> blocks  = 0
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of 
> under-replicated blocks = 0
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of  
> over-replicated blocks = 0
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of blocks 
> being written= 0
> 2012-08-06 15:09:46,383 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Will take over writing 
> edit logs at txnid 2354
> 2012-08-06 15:09:46,471 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 2354
> 2012-08-06 15:09:46,472 FATAL 
> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: starting log segment 
> 2354 failed for required journal 
> (JournalAndStream(mgr=org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager@4eda1515,
>  stream=null))
> java.io.IOException: We've already seen 2354. A new stream cannot be created 
> with it
> at 
> org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.startLogSegment(BookKeeperJournalManager.java:297)
> at 
> org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalAndStream.startLogSegment(JournalSet.java:86)
> at 
> org.apache.hadoop.hdfs.server.namenode.JournalSet$2.apply(JournalSet.java:182)
> at 
> org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:319)
> at 
> org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:179)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:894)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSEditLog.openForWrite(FSEditLog.java:268)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:618)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:1322)
> at 
> org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:61)
> at 
> org.apache.hadoop.hdfs.server.namenode.ha.HAState.setStateInternal(HAState.java:63)
> at 
> org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.setState(StandbyState.java:49)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.transitionToActive(NameNode.java:1230)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.transitionToActive(NameNodeRpcServer.java:990)
> at 
> org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.transitionToActive(HAServiceProtocolServerSideTranslatorPB.java:107)
> at 
> org.apache.hadoop.ha.p