Re: Fwd: Spark SQL: ArrayIndexOutofBoundsException

2016-11-24 Thread cossy
drop() function is in scala,an attribute of Array,no in spark -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-ArrayIndexOutofBoundsException-tp15639p28127.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: Spark SQL ArrayIndexOutOfBoundsException

2015-05-12 Thread Michael Armbrust
.. > p(770),p(771),p(772),p(773),p(774),p(775),p(776),p(777),p(778),p(779), > p(780),p(781),p(782),p(783),p(784) > > i.e by specifying all 785 elements physically > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n

Re: Spark SQL ArrayIndexOutOfBoundsException

2015-05-11 Thread mikejf12
p(14),p(15),p(16),p(17),p(18),p(19), . p(770),p(771),p(772),p(773),p(774),p(775),p(776),p(777),p(778),p(779), p(780),p(781),p(782),p(783),p(784) i.e by specifying all 785 elements physically -- View this message in context: http://apache-spark-user-list.1001560.n3.nabb

Spark SQL ArrayIndexOutOfBoundsException

2015-05-11 Thread Mike Frampton
Sorry if this email is a duplicate, I realised that I was not registered with the mailing list ... I am having a problem with a spark sql script which is running on a spark 1.2 CentOS CDH 5.3 mini 5 node cluster. The script processes some image csv data each record/line of which has 28x28 i

Re: Fwd: Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread Michael Armbrust
as a >> suggestion. Currently I am just deleting this header line manually before >> processing it in Spark. >> >> >> thanks >> >> >> >> >> >> -- >> View this message in c

Re: Fwd: Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread Sunny Khatri
s message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-ArrayIndexOutofBoundsException-tp15639p15642.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > -

Re: Fwd: Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread SK
just deleting this header line manually before processing it in Spark. thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-ArrayIndexOutofBoundsException-tp15639p15642.html Sent from the Apache Spark User List mailing list archive at

Fwd: Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread Liquan Pei
-- Forwarded message -- From: Liquan Pei Date: Thu, Oct 2, 2014 at 3:42 PM Subject: Re: Spark SQL: ArrayIndexOutofBoundsException To: SK There is only one place you use index 1. One possible issue is that your may have only one element after your split by "\t". Can

Re: Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread Michael Armbrust
// register the RDD as a table > tusers.registerTempTable("tusers") > > // get the number of unique users > val unique_count = sql_cxt.sql("SELECT COUNT (DISTINCT userid) FROM > tusers").collect().head.getLong(0) > > println(unique_count

Spark SQL: ArrayIndexOutofBoundsException

2014-10-02 Thread SK
s val unique_count = sql_cxt.sql("SELECT COUNT (DISTINCT userid) FROM tusers").collect().head.getLong(0) println(unique_count) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-ArrayIndexOutofBoundsException-tp15639.html Sent from t