Did you build Spark with: -Pnetlib-lgpl? Ref: https://spark.apache.org/docs/latest/mllib-guide.html
Burak On Sun, Mar 22, 2015 at 7:37 AM, Ted Yu <[email protected]> wrote: > How about pointing LD_LIBRARY_PATH to native lib folder ? > > You need Spark 1.2.0 or higher for the above to work. See SPARK-1719 > > Cheers > > On Sun, Mar 22, 2015 at 4:02 AM, Xi Shen <[email protected]> wrote: > >> Hi Ted, >> >> I have tried to invoke the command from both cygwin environment and >> powershell environment. I still get the messages: >> >> 15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: >> com.github.fommil.netlib.NativeSystemBLAS >> 15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: >> com.github.fommil.netlib.NativeRefBLAS >> >> From the Spark UI, I can see: >> >> spark.driver.extraLibrary c:\openblas >> >> >> Thanks, >> David >> >> >> On Sun, Mar 22, 2015 at 11:45 AM Ted Yu <[email protected]> wrote: >> >>> Can you try the --driver-library-path option ? >>> >>> spark-submit --driver-library-path /opt/hadoop/lib/native ... >>> >>> Cheers >>> >>> On Sat, Mar 21, 2015 at 4:58 PM, Xi Shen <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> I use the *OpenBLAS* DLL, and have configured my application to work >>>> in IDE. When I start my Spark application from IntelliJ IDE, I can see in >>>> the log that the native lib is loaded successfully. >>>> >>>> But if I use *spark-submit* to start my application, the native lib >>>> still cannot be load. I saw the WARN message that it failed to load both >>>> the native and native-ref library. I checked the *Environment* tab in >>>> the Spark UI, and the *java.library.path* is set correctly. >>>> >>>> >>>> Thanks, >>>> >>>> David >>>> >>>> >>>> >>> >
