Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-25 Thread Akhil Das
Try deleting the .iv2 directory in your home and then do a sbt clean assembly would solve this issue i guess. Thanks Best Regards On Thu, Jun 26, 2014 at 3:10 AM, Robert James wrote: > In case anyone else is having this problem, deleting all ivy's cache, > then doing a sbt clean, then recompil

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-25 Thread Robert James
In case anyone else is having this problem, deleting all ivy's cache, then doing a sbt clean, then recompiling everything, repackaging, and reassemblying, seems to have solved the problem. (From the sbt docs, it seems that having to delete ivy's cache means a bug in sbt) On 6/25/14, Robert James

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-25 Thread Robert James
Thanks Paul. I'm unable to follow the discussion on SPARK-2075. But how would you recommend I test or follow up on that? Is there a workaround? On 6/25/14, Paul Brown wrote: > Hi, Robert -- > > I wonder if this is an instance of SPARK-2075: > https://issues.apache.org/jira/browse/SPARK-2075 > >

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-25 Thread Paul Brown
Hi, Robert -- I wonder if this is an instance of SPARK-2075: https://issues.apache.org/jira/browse/SPARK-2075 -- Paul — p...@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/ On Wed, Jun 25, 2014 at 6:28 AM, Robert James wrote: > On 6/24/14, Robert James wrote: > > My app works f

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-25 Thread Robert James
On 6/24/14, Robert James wrote: > My app works fine under Spark 0.9. I just tried upgrading to Spark > 1.0, by downloading the Spark distro to a dir, changing the sbt file, > and running sbt assembly, but I get now NoSuchMethodErrors when trying > to use spark-submit. > > I copied in the SimpleAp

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-24 Thread Robert James
On 6/24/14, Peng Cheng wrote: > I got 'NoSuchFieldError' which is of the same type. its definitely a > dependency jar conflict. spark driver will load jars of itself which in > recent version get many dependencies that are 1-2 years old. And if your > newer version dependency is in the same packag

Re: Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-24 Thread Peng Cheng
I got 'NoSuchFieldError' which is of the same type. its definitely a dependency jar conflict. spark driver will load jars of itself which in recent version get many dependencies that are 1-2 years old. And if your newer version dependency is in the same package it will be shaded (Java's first come

Upgrading to Spark 1.0.0 causes NoSuchMethodError

2014-06-24 Thread Robert James
My app works fine under Spark 0.9. I just tried upgrading to Spark 1.0, by downloading the Spark distro to a dir, changing the sbt file, and running sbt assembly, but I get now NoSuchMethodErrors when trying to use spark-submit. I copied in the SimpleApp example from http://spark.apache.org/docs/