Hi, After some amount of debugging, I narrowed down the problem. I'd set multiple paths (multiple disks mounted at different endpoints) for the * mapred.local.dir* setting. When I changed this to just one path, everything started working - previously the job jar and the job xml weren't being created under the local/taskTracker/jobCache folder.
Any idea why? Am I doing something wrong? I've read somewhere else that specifying multiple disks for mapred.local.dir is important to increase disk bandwidth so that the map outputs get written faster to the local disks on the tasktracker nodes. Cheers, Harish On Tue, Jun 9, 2009 at 5:21 PM, Harish Mallipeddi < [email protected]> wrote: > I setup a new cluster (1 namenode + 2 datanodes). I'm trying to run the > GenericMRLoadGenerator program from hadoop-0.20.0-test.jar. But I keep > getting the ClassNotFoundException. Any reason why this would be happening? > It seems to me like the tasktrackers cannot find the class files from the > hadoop program but when you do "hadoop jar", it will automatically ship the > job jar file to all the tasktracker nodes and put them in the classpath? > > $ *bin/hadoop jar hadoop-0.20.0-test.jar > loadgen*-Dtest.randomtextwrite.bytes_per_map=1048576 > -Dhadoop.sort.reduce.keep.percent=50.0 -outKey org.apache.hadoop.io.Text > -outValue org.apache.hadoop.io.Text > > No input path; ignoring InputFormat > Job started: Tue Jun 09 03:16:30 PDT 2009 > 09/06/09 03:16:30 INFO mapred.JobClient: Running job: job_200906090316_0001 > 09/06/09 03:16:31 INFO mapred.JobClient: map 0% reduce 0% > 09/06/09 03:16:40 INFO mapred.JobClient: Task Id : > attempt_200906090316_0001_m_000000_0, Status : FAILED > java.io.IOException: Split class > org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit > not found > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:324) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > *Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit > * > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:761) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:321) > ... 2 more > [more of the same exception from different map tasks...I've removed them] > > Actually the same thing happens even if I run the PiEstimator program from > hadoop-0.20.0-examples.jar. > > Thanks, > > -- > Harish Mallipeddi > http://blog.poundbang.in > -- Harish Mallipeddi http://blog.poundbang.in
