Wu Wei wrote:
Hi,
I used to submit Hadoop job with the utility RunJar.main() on hadoop
0.18. On hadoop 0.19, because the commandLineConfig of JobClient was
null, I got a NullPointerException error when RunJar.main() calls
GenericOptionsParser to get libJars (0.18 didn't do this call). I also
tried the class JobShell to submit job, but it catches all exceptions
and sends to stderr so that I cann't handle the exceptions myself.
I noticed that if I can call JobClient's setCommandLineConfig method,
everything goes easy. But this method has default package accessibility,
I cann't see the method out of package org.apache.hadoop.mapred.
Any advices on using Java APIs to submit job?
Wei
Looking at my code, the line that does the work is
JobClient jc = new JobClient(jobConf);
runningJob = jc.submitJob(jobConf);
My full (LGPL) code is here : http://tinyurl.com/djk6vj
there's more work with validating input and output directories, pulling
back the results, handling timeouts if the job doesnt complete, etc,etc,
but that's feature creep
