Yep that was it. I'd a whitespace after the comma. Thanks Aaron! On Wed, Jun 10, 2009 at 9:39 PM, Aaron Kimball <[email protected]> wrote:
> mapred.local.dir can definitely work with multiple paths, but it's a bit > strict about the format. You should have one or more paths, separated by > commas and no whitespace. e.g. "/disk1/mapred,/disk2/mapred". If you've got > them on new lines, etc, then it might try to interpret that as part of one > of the paths. Could this be your issue? If you paste your hadoop-site.xml > in > to an email we could take a look. > > - Aaron > > On Wed, Jun 10, 2009 at 2:41 AM, Harish Mallipeddi < > [email protected]> wrote: > > > Hi, > > > > After some amount of debugging, I narrowed down the problem. I'd set > > multiple paths (multiple disks mounted at different endpoints) for the * > > mapred.local.dir* setting. When I changed this to just one path, > everything > > started working - previously the job jar and the job xml weren't being > > created under the local/taskTracker/jobCache folder. > > > > Any idea why? Am I doing something wrong? I've read somewhere else that > > specifying multiple disks for mapred.local.dir is important to increase > > disk > > bandwidth so that the map outputs get written faster to the local disks > on > > the tasktracker nodes. > > > > Cheers, > > Harish > > > > On Tue, Jun 9, 2009 at 5:21 PM, Harish Mallipeddi < > > [email protected]> wrote: > > > > > I setup a new cluster (1 namenode + 2 datanodes). I'm trying to run the > > > GenericMRLoadGenerator program from hadoop-0.20.0-test.jar. But I keep > > > getting the ClassNotFoundException. Any reason why this would be > > happening? > > > It seems to me like the tasktrackers cannot find the class files from > the > > > hadoop program but when you do "hadoop jar", it will automatically ship > > the > > > job jar file to all the tasktracker nodes and put them in the > classpath? > > > > > > $ *bin/hadoop jar hadoop-0.20.0-test.jar > > loadgen*-Dtest.randomtextwrite.bytes_per_map=1048576 > > > -Dhadoop.sort.reduce.keep.percent=50.0 -outKey > org.apache.hadoop.io.Text > > > -outValue org.apache.hadoop.io.Text > > > > > > No input path; ignoring InputFormat > > > Job started: Tue Jun 09 03:16:30 PDT 2009 > > > 09/06/09 03:16:30 INFO mapred.JobClient: Running job: > > job_200906090316_0001 > > > 09/06/09 03:16:31 INFO mapred.JobClient: map 0% reduce 0% > > > 09/06/09 03:16:40 INFO mapred.JobClient: Task Id : > > > attempt_200906090316_0001_m_000000_0, Status : FAILED > > > java.io.IOException: Split class > > > > > > org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit > > > not found > > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:324) > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) > > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > *Caused by: java.lang.ClassNotFoundException: > > > > > > org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit > > > * > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > > > at java.security.AccessController.doPrivileged(Native Method) > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) > > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > > > at java.lang.Class.forName0(Native Method) > > > at java.lang.Class.forName(Class.java:247) > > > at > > > > > > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:761) > > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:321) > > > ... 2 more > > > [more of the same exception from different map tasks...I've removed > them] > > > > > > Actually the same thing happens even if I run the PiEstimator program > > from > > > hadoop-0.20.0-examples.jar. > > > > > > Thanks, > > > > > > -- > > > Harish Mallipeddi > > > http://blog.poundbang.in > > > > > > > > > > > -- > > Harish Mallipeddi > > http://blog.poundbang.in > > > -- Harish Mallipeddi http://blog.poundbang.in
