We removed commons-math3 from dependencies to avoid version conflict with hadoop-common. hadoop-common-2.3+ depends on commons-math3-3.1.1, while breeze depends on commons-math3-3.3. 3.3 is not backward compatible with 3.1.1. So we removed it because the breeze functions we use do not touch commons-math3 code. As Sean suggested, please include breeze in the dependency set of your project. Do not rely on transitive dependencies. -Xiangrui
On Fri, Sep 26, 2014 at 9:08 AM, Jaonary Rabarisoa <[email protected]> wrote: > I solve the problem by including the commons-math3 package in my sbt > dependencies as Sean suggested. Thanks. > > On Fri, Sep 26, 2014 at 6:05 PM, Ted Yu <[email protected]> wrote: >> >> You can use scope of runtime. >> >> See >> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope >> >> Cheers >> >> On Fri, Sep 26, 2014 at 8:57 AM, Jaonary Rabarisoa <[email protected]> >> wrote: >>> >>> Thank Ted. Can you tell me how to adjust the scope ? >>> >>> On Fri, Sep 26, 2014 at 5:47 PM, Ted Yu <[email protected]> wrote: >>>> >>>> spark-core's dependency on commons-math3 is @ test scope (core/pom.xml): >>>> <dependency> >>>> <groupId>org.apache.commons</groupId> >>>> <artifactId>commons-math3</artifactId> >>>> <version>3.3</version> >>>> <scope>test</scope> >>>> </dependency> >>>> >>>> Adjusting the scope should solve the problem below. >>>> >>>> On Fri, Sep 26, 2014 at 8:42 AM, Jaonary Rabarisoa <[email protected]> >>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I'm using some functions from Breeze in a spark job but I get the >>>>> following build error : >>>>> >>>>> Error:scalac: bad symbolic reference. A signature in RandBasis.class >>>>> refers to term math3 >>>>> in package org.apache.commons which is not available. >>>>> It may be completely missing from the current classpath, or the version >>>>> on >>>>> the classpath might be incompatible with the version used when >>>>> compiling RandBasis.class. >>>>> >>>>> In my case, I just declare a new Gaussian distribution >>>>> >>>>> val g = new Gaussian(0d,1d) >>>>> >>>>> I'm using spark 1.1 >>>>> >>>>> >>>>> Any ideas to fix this ? >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> >>>>> Jao >>>> >>>> >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
