Hi!
I’ve been experimenting with the SSVDSolver and unfortunately, during runtime,
I encounter this error:
10648576 [Thread-13] WARN org.apache.hadoop.mapred.LocalJobRunner -
job_local1958711697_0001
java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:546)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.httpclient.HttpMethod
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
Exception in thread "Thread-13" java.lang.NoClassDefFoundError:
org/apache/commons/httpclient/HttpMethod
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:562)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.httpclient.HttpMethod
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
The actual invocation is:
public static void runSSVDOnSparseVectors(String inputPath,
String outputPath, int rank, int oversampling, int
blocks,
int reduceTasks, int powerIterations, boolean halfSigma)
throws IOException {
Configuration conf = new Configuration();
SSVDSolver solver = new SSVDSolver(conf, new Path[] { new Path(
inputPath) }, new Path(outputPath), blocks, rank,
oversampling,
reduceTasks);
solver.setQ(powerIterations);
if (halfSigma) {
solver.setcUHalfSigma(true);
solver.setcVHalfSigma(true);
}
solver.run();
}
while being invoked with (input.getParent() + “/" + TERM_DOC_MATRIX_NAME,
input.getParent() + “/" + SVD_FOLDER_NAME, k, 2 * k, Math.min(200000, (int) (3
* k * 0.01 * Math.max(lsaTraining.getNoDocuments(),lsaTraining.getNoWords()))),
5, 2, true);
I’m using Mahout 0.10 with httpclient-4.4.1.jar (I tried also 4.2.5 from the
package archive) on a 48k words X 53k docs matrix.
Any ideas? It works fine with the similar variables if I run the job in command
line.
Also, how should I tweak the input variables?
Thanks in advance!
Mihai
