Hi, everyone,
I’m trying to do some performance test on HDFS with filebench which comes with
hadoop-0.19.0-test.jar.
I successfully run the command with –r attribute (read testing) and get the
output.
(the command :hadoop jar hadoop-0.19.0-test.jar filebench -r -txt -blk -pln
-dir "hdfs://master:54310/dir1")
But failed to run with –w attribute (write testing).
The command and error message are below, any suggestions will be appreciated.
BTW, is there any document about hadoop-xxx-test.jar? It seems I can only get
information from tracing the source code?
Regards,
Hsiao-Feng, Yeh
===============================
[had...@164-22 hadoop]$ hadoop jar hadoop-0.19.0-test.jar filebench -w -txt
-blk -pln -dir "hdfs://master:54310/dir1"
DIR: hdfs://master:54310/dir1
W TXT_PLN: java.lang.NullPointerException
at
org.apache.hadoop.mapred.FileOutputCommitter.getWorkPath(FileOutputCommitter.java:201)
at
org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:227)
at
org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:115)
at org.apache.hadoop.io.FileBench.writeBench(FileBench.java:113)
at org.apache.hadoop.io.FileBench$RW$1.exec(FileBench.java:295)
at org.apache.hadoop.io.FileBench.run(FileBench.java:235)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.io.FileBench.main(FileBench.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:141)
at org.apache.hadoop.test.AllTestDriver.main(AllTestDriver.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
