Thanks very much for the above. In 0.21 I found TestDFSIO in the
mapred-test.jar, but I didn't find Teragen and Terasort in the three test
jars (I found Teragen and Terasort in 0.20.04 test jars, but they cannot be
executed in 0.21). When I run TestDFSIO benchmark, I sometimes encounter
the situati
Once you have an instrumented cluster you can run some end-to-end tests
(TeraSort
has been suggested and it would do fine). The tricky part is to assemble all
moving parts of the Hadoop cluster together from three different projects (if
you are on .21+ branch) and will be a way easier for 0.20.+ b
I'd add that one thing I'd like is better low level hardware fault
simulation, something that doesn't inject faults into the software, so
much as stamp on the underlying data or kill bits of the stack entirely
That is
-deliberately corrupting blocks or checksums
-deleting blocks
-dropping
Hi Hao,
You may want to setup a full Hadoop 'one box' install and start running the
TestDFSIO, Teragen and Terasort (with a small amount of data) scenarios using
the injected jar to run hdfs. That may be the fastest and best way to get up
and running.
When it comes to fault injection the valu
