Hello,
I would like to run a mapreduce program on hadoop 0.19.0. The program is
designed to iterate for 1000 times. However, in a certain iteration, the
program fails due to an EOFException, but the jobs complete for all the
previous iterations. Sometimes the program fails in the 5th iteration, while in
other times, the program might fail until the 9th iteration. It seems a little
random.
A similar situation has been discussed for hadoop version 0.2, but it is said
the bug has been fixed in version 0.3. Anybody knows that what is wrong with my
hadoop 0.19.0?
...
09/01/04 12:54:37 INFO mapred.JobClient: map 60% reduce 0%
09/01/04 12:54:41 INFO mapred.JobClient: map 66% reduce 0%
09/01/04 12:54:41 INFO mapred.JobClient: Task Id :
attempt_200901041244_0007_m_000011_0, Status : FAILED
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:43 INFO mapred.JobClient: map 70% reduce 1%
09/01/04 12:54:45 INFO mapred.JobClient: map 78% reduce 1%
09/01/04 12:54:46 INFO mapred.JobClient: map 81% reduce 2%
09/01/04 12:54:48 INFO mapred.JobClient: map 83% reduce 2%
09/01/04 12:54:48 INFO mapred.JobClient: Task Id :
attempt_200901041244_0007_m_000011_1, Status : FAILED
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:50 INFO mapred.JobClient: map 83% reduce 3%
09/01/04 12:54:51 INFO mapred.JobClient: map 86% reduce 4%
09/01/04 12:54:53 INFO mapred.JobClient: map 86% reduce 6%
09/01/04 12:54:53 INFO mapred.JobClient: Task Id :
attempt_200901041244_0007_m_000011_2, Status : FAILED
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:55 INFO mapred.JobClient: map 86% reduce 9%
09/01/04 12:54:56 INFO mapred.JobClient: map 88% reduce 10%
09/01/04 12:54:58 INFO mapred.JobClient: map 88% reduce 15%
java.io.IOException: Job failed!
Thanks a lot!
2009-01-04
gziyang
