Re: Code works in Spark-Shell but Fails inside IntelliJ

2014-11-20 Thread Sanjay Subramanian
s" to quickly test , experiment and debug code. From: Jay Vyas To: Sanjay Subramanian Cc: "user@spark.apache.org" Sent: Thursday, November 20, 2014 4:53 PM Subject: Re: Code works in Spark-Shell but Fails inside IntelliJ This seems pretty standard: your IntelliJ classp

Re: Code works in Spark-Shell but Fails inside IntelliJ

2014-11-20 Thread Sanjay Subramanian
Subramanian Cc: "user@spark.apache.org" Sent: Thursday, November 20, 2014 4:49 PM Subject: Re: Code works in Spark-Shell but Fails inside IntelliJ Looks like intelij might be trying to load the wrong version of spark? On Thu, Nov 20, 2014 at 4:35 PM, Sanjay Subramanian wrote:

Re: Code works in Spark-Shell but Fails inside IntelliJ

2014-11-20 Thread Jay Vyas
This seems pretty standard: your IntelliJ classpath isn't matched to the correct ones that are used in spark shell Are you using the SBT plugin? If not how are you putting deps into IntelliJ? > On Nov 20, 2014, at 7:35 PM, Sanjay Subramanian > wrote: > > hey guys > > I am at AmpCamp 2014

Re: Code works in Spark-Shell but Fails inside IntelliJ

2014-11-20 Thread Michael Armbrust
Looks like intelij might be trying to load the wrong version of spark? On Thu, Nov 20, 2014 at 4:35 PM, Sanjay Subramanian < sanjaysubraman...@yahoo.com.invalid> wrote: > hey guys > > I am at AmpCamp 2014 at UCB right now :-) > > Funny Issue... > > This code works in Spark-Shell but throws a funn

Code works in Spark-Shell but Fails inside IntelliJ

2014-11-20 Thread Sanjay Subramanian
hey guys I am at AmpCamp 2014 at UCB right now :-)  Funny Issue... This code works in Spark-Shell but throws a funny exception in IntelliJ CODE val sqlContext = new org.apache.spark.sql.SQLContext(sc)sqlContext.setConf("spark.sql.parquet.binaryAsString", "true")val wikiData = sqlContext.parq