Hi Jikai,
The reason I ask is because your stacktrace has this section in it:
com.hadoop.compression.lzo.GPLNativeCodeLoader.(
GPLNativeCodeLoader.java:32)
at com.hadoop.compression.lzo.LzoCodec.(LzoCodec.java:71)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(C
Thanks Andrew. Actually my job did not use any data in .lzo format. Here is
the program itself:
import org.apache.spark._
import org.apache.spark.mllib.util.MLUtils
import org.apache.spark.mllib.classification.LogisticRegressionWithSGD
object Test {
def main(args: Array[String]) {
val spar
Thanks. I tried this option, but still got the same error.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Could-not-load-native-gpl-library-tp11743p11791.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
Hi Jikai,
It looks like you're trying to run a Spark job on data that's stored in
HDFS in .lzo format. Spark can handle this (I do it all the time), but you
need to configure your Spark installation to know about the .lzo format.
There are two parts to the hadoop lzo library -- the first is the
Is the GPL library only available on the driver node? If that is the
case, you need to add them to `--jars` option of spark-submit.
-Xiangrui
On Thu, Aug 7, 2014 at 6:59 PM, Jikai Lei wrote:
> I had the following error when trying to run a very simple spark job (which
> uses logistic regression w
