Just wanted to add more info.. I was using SparkSQL reading in the
tab-delimited raw data files converting the timestamp to Date format:
sc.textFile("rawdata/*").map(_.split("\t")).map(p => Point(df.format(new
Date( p(0).trim.toLong*1000L )), p(1), p(2).trim.toInt ,p(3).trim.toInt,
p(4).trim.toInt ,p(5)))
Then I go about registering it as table and when I run simple query like
select count(*) from
, I get the ArrayIndexOutOfBoundsException.
I bumped up the SPARK_DRIVER_MEMORY to 8g but still didn't help:
export SPARK_DRIVER_MEMORY=8g
Let me know if I'm missing any steps.. thanks!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-shell-running-into-ArrayIndexOutOfBoundsException-tp10480p10520.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
