Re: Ivy support in Spark vs. sbt

2015-06-07 Thread Igor Costa
Marcelo I've run this problem once, when I was starting with Spark, like you mentioned. I found out that ivy get messy with diff sbt version. My solution was using a previous compatible version with sbt to not cross with version. Best Igor On Thu, Jun 4, 2015 at 8:08 PM, Eron Wright wrote: >

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Eron Wright
I saw something like this last night, with a similar message. Is this what you’re referring to? [error] org.deeplearning4j#dl4j-spark-ml;0.0.3.3.4.alpha1-SNAPSHOT!dl4j-spark-ml.jar origin location must be absolute: file:/Users/eron/.m2/repository/org/deeplearning4j/dl4j-spark-ml/0.0.3.3.4.alp

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Marcelo Vanzin
Here's one of the types of exceptions I get (this one when running VersionsSuite from sql/hive): [info] - 13: create client *** FAILED *** (1 second, 946 milliseconds) [info] java.lang.RuntimeException: [download failed: org.apache.httpcomponents#httpclient;4.2.5!httpclient.jar, download failed:

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread shane knapp
interesting... i definitely haven't seen it happen that often in our build system, and when it has happened, i wasn't able to determine the cause. On Thu, Jun 4, 2015 at 10:16 AM, Marcelo Vanzin wrote: > On Thu, Jun 4, 2015 at 10:04 AM, shane knapp wrote: > >> this has occasionally happened on

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Marcelo Vanzin
On Thu, Jun 4, 2015 at 10:04 AM, shane knapp wrote: > this has occasionally happened on our jenkins as well (twice since last > august), and deleting the cache fixes it right up. > Yes deleting the cache fixes things, but it's kinda annoying to have to do that. And yesterday when I was testing a

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread shane knapp
this has occasionally happened on our jenkins as well (twice since last august), and deleting the cache fixes it right up. On Thu, Jun 4, 2015 at 4:29 AM, Sean Owen wrote: > I've definitely seen the "dependency path must be relative" problem, > and fixed it by deleting the ivy cache, but I don't

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Marcelo Vanzin
They're my local builds, so I wouldn't be able to send you any links... and the error is generally from sbt, not the unit tests. But if there's any info I can collect when I see the error, let me know. I'll try "spark.jars.ivy". I wonder if we should just set that to the system properties in Spark

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Burak Yavuz
Hi Marcelo, This is interesting. Can you please send me links to any failing builds if you see that problem please. For now you can set a conf: `spark.jars.ivy` to use a path except `~/.ivy2` for Spark. Thanks, Burak On Thu, Jun 4, 2015 at 4:29 AM, Sean Owen wrote: > I've definitely seen the "

Re: Ivy support in Spark vs. sbt

2015-06-04 Thread Sean Owen
I've definitely seen the "dependency path must be relative" problem, and fixed it by deleting the ivy cache, but I don't know more than this. On Thu, Jun 4, 2015 at 1:33 AM, Marcelo Vanzin wrote: > Hey all, > > I've been bit by something really weird lately and I'm starting to think > it's relate

Ivy support in Spark vs. sbt

2015-06-03 Thread Marcelo Vanzin
Hey all, I've been bit by something really weird lately and I'm starting to think it's related to the ivy support we have in Spark, and running unit tests that use that code. The first thing that happens is that after running unit tests, sometimes my sbt builds start failing with error saying som