Hadoop-Hdfs-trunk - Build # 1094 - Still Failing

2012-07-05 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1094/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 11638 lines...]
[INFO] 
[INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default) @ hadoop-hdfs-raid ---
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is true
[INFO] ** FindBugsMojo executeFindbugs ***
[INFO] Temp File is 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs-raid/target/findbugsTemp.xml
[INFO] Fork Value is true
 [java] Warnings generated: 31
[INFO] xmlOutput is false
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (site) @ hadoop-hdfs-raid ---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs-raid/target/docs-src
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  SUCCESS [5:21.929s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [35.324s]
[INFO] Apache Hadoop HDFS BookKeeper Journal . SUCCESS [13.880s]
[INFO] Apache Hadoop HDFS Raid ... FAILURE [18.153s]
[INFO] Apache Hadoop HDFS Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 6:30.108s
[INFO] Finished at: Thu Jul 05 11:39:23 UTC 2012
[INFO] Final Memory: 57M/720M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-antrun-plugin:1.6:run (site) on project 
hadoop-hdfs-raid: An Ant BuildException has occured: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs-raid/src/main/docs
 does not exist. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs-raid
Build step 'Execute shell' marked build as failure
Archiving artifacts
Updating HDFS-3575
Updating HDFS-3574
Updating HADOOP-8110
Updating HDFS-3573
Updating HDFS-3190
Updating MAPREDUCE-4371
Updating HADOOP-8434
Updating HADOOP-8463
Updating HDFS-3601
Updating HADOOP-8533
Updating HDFS-3343
Updating HADOOP-8548
Updating HDFS-3581
Updating HADOOP-8547
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1094

2012-07-05 Thread Apache Jenkins Server 
See 

Changes:

[szetszwo] HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block 
placement with 4-layer network topology.  Contributed by Junping Du

[eli] HDFS-3581. FSPermissionChecker#checkPermission sticky bit check missing 
range check. Contributed by Eli Collins

[eli] HADOOP-8547. Package hadoop-pipes examples/bin directory (again). 
Contributed by Colin Patrick McCabe

[todd] HDFS-3574. Fix small race and do some cleanup in GetImageServlet. 
Contributed by Todd Lipcon.

[todd] HDFS-3343. Improve metrics for DN read latency. Contributed by Andrew 
Wang.

[daryn] HADOOP-8110. Fix trash checkpoint collisions (Jason Lowe via daryn)

[suresh] HADOOP-8434. Add tests for Configuration setter methods. Contributed 
by Madhukara Phatak.

[suresh] Moving HADOOP-8533 to release 2.0 section

[todd] HDFS-3190. Simple refactors in existing NN code to assist 
QuorumJournalManager extension. Contributed by Todd Lipcon.

[eli] HADOOP-8463. hadoop.security.auth_to_local needs a key definition and 
doc. Contributed by Madhukara Phatak

[suresh] HADOOP-8533. Remove parallel call ununsed capability in RPC. 
Contributed by Brandon Li.

[bobby] HADOOP-8548. test-patch.sh shows an incorrect link in Jekins builds 
(Kihwal Lee via bobby)

[bobby] MAPREDUCE-4371. Check for cyclic dependencies in Jobcontrol job DAG 
(madhukara phatak via bobby)

[todd] HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers. 
Contributed by Todd Lipcon.

[tucu] HDFS-3575. HttpFS does not log Exception Stacktraces (brocknoland via 
tucu)

--
[...truncated 11445 lines...]
[INFO] --- maven-checkstyle-plugin:2.6:checkstyle (default-cli) @ 
hadoop-hdfs-httpfs ---
[INFO] 
[INFO] There are 1598 checkstyle errors.
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] 
[INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default-cli) @ 
hadoop-hdfs-httpfs ---
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is true
[INFO] ** FindBugsMojo executeFindbugs ***
[INFO] Temp File is 

[INFO] Fork Value is true
[INFO] xmlOutput is false
[INFO] 
[INFO] 
[INFO] Building Apache Hadoop HDFS BookKeeper Journal 3.0.0-SNAPSHOT
[INFO] 
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
no dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: Failed to read artifact descriptor for 
org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ 
hadoop-hdfs-bkjournal ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ 
hadoop-hdfs-bkjournal ---
[INFO] Executing tasks

main:
[mkdir] Created dir: 

[INFO] Executed tasks
[INFO] 
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @ 
hadoop-hdfs-bkjournal ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ 
hadoop-hdfs-bkjournal ---
[INFO] Using default encoding to copy filtered resources.
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
hadoop-hdfs-bkjournal ---
[INFO] Compiling 6 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ 
hadoop-hdfs-bkjournal ---
[INFO] Using default encoding to copy filtered resources.
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
hadoop-hdfs-bkjournal ---
[INFO] Compiling 8 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
hadoop-hdfs-bkjournal ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-hdfs-bkjournal ---
[INFO] Building jar:

[jira] [Created] (HDFS-3602) Enhancements to HDFS for Windows Server and Windows Azure development and runtime environments

2012-07-05 Thread Bikas Saha (JIRA) 
Bikas Saha created HDFS-3602:


 Summary: Enhancements to HDFS for Windows Server and Windows Azure 
development and runtime environments
 Key: HDFS-3602
 URL: https://issues.apache.org/jira/browse/HDFS-3602
 Project: Hadoop HDFS
  Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: Bikas Saha
Assignee: Bikas Saha


This JIRA tracks the work that needs to be done on trunk to enable Hadoop to 
run on Windows Server and Azure environments. This incorporates porting 
relevant work from the similar effort on branch 1 tracked via HADOOP-8079.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3603) TestHDFSTrash is failing

2012-07-05 Thread Jason Lowe (JIRA) 
Jason Lowe created HDFS-3603:


 Summary: TestHDFSTrash is failing
 Key: HDFS-3603
 URL: https://issues.apache.org/jira/browse/HDFS-3603
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: test
Affects Versions: 0.23.3, 2.0.1-alpha, 3.0.0
Reporter: Jason Lowe
Priority: Blocker


TestHDFSTrash is failing pretty regularly during test builds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3604) Add dfs.webhdfs.enabled to hdfs-default.xml

2012-07-05 Thread Eli Collins (JIRA) 
Eli Collins created HDFS-3604:
-

 Summary: Add dfs.webhdfs.enabled to hdfs-default.xml
 Key: HDFS-3604
 URL: https://issues.apache.org/jira/browse/HDFS-3604
 Project: Hadoop HDFS
  Issue Type: Improvement
Affects Versions: 2.0.0-alpha, 1.0.0
Reporter: Eli Collins
Assignee: Eli Collins
Priority: Minor


Let's add {{dfs.webhdfs.enabled}} to hdfs-default.xml.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: transfer large file onto hdfs

2012-07-05 Thread Harsh J 
Should we assume that the non-hadoop system has no way to get on the
network of the hadoop cluster and its clients? Otherwise, you can
grant it temporary access and do a write from it itself. If not:

Run a micro FTP server pointed to that file, and then do a 'hadoop fs
-cp ftp://location hdfs://location', since FTPFileSystem is present in
Hadoop? Or if NFS/etc. mounted, file:/// will work (or via
copyFromLocal/put). Essentially you're bringing in the file remotely
but performing the copy via CLI.

Or you can copy them in chunks, either keeping the destination file
writer open, if possible, or appending (depending on what version of
Hadoop you're using).

On Thu, Jul 5, 2012 at 11:54 PM, Ted Yu  wrote:
> Hi,
> One of the customers wants to transfer dump file (size ~ 2TB) from outside
> hadoop cluster onto hdfs.
> The size exceeds free space on CLI machine.
>
> I want to poll best practice in this scenario.
>
> Thanks



-- 
Harsh J