Hi Guys,
recently I upgraded to the recent Claudera Hadoop distribution. It contains
hadoop-core-0.20.2+737.jar. If I now run my map job, I
get the following exception for a few tasks:
java.io.IOException: Owner 'mapred' for path
/hadoop/hdfs5/tmp/taskTracker/mathias.walter/jobcache/job_201010210928_0005/attempt_201010210928_0005_m_000000_0/output/spill437.out
.index did not match expected owner 'mathias.walter'
at org.apache.hadoop.io.SecureIOUtils.checkStat(SecureIOUtils.java:182)
at
org.apache.hadoop.io.SecureIOUtils.openForRead(SecureIOUtils.java:108)
at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:62)
at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:55)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1480)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1172)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:574)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:641)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
A total of 8 tasks are running in parallel. They are finished after about 8
hours, but some of them (19) were crashed with the above
exception.
Why are so many tasks crashed, but some not?
--
Kind regards,
Mathias
