Can you verify that the path that the JVM tries to find the library
(present in the LinkageError) is valid and is a part of java.library.path ?
You can do this by printing `(System.getProperty("java.library.path"));`
inside your StreamTask's *init* method.
Best,
Jagadish
On Thu, Jun 8, 2017 at 9:22 PM, Angelica Garcia-Gutierrez <
agar...@transcendinsights.com> wrote:
> Thanks for the hint!
> I made some progress.
>
> By adding task.opts to the job configuration file the JVM was able to find
> the library cql_engine12d. However, this library depends on some other
> libraries e.g. cql_types_12d (located in the same directory as
> cql_engine12d) and am hitting the following error:
>
> Java HotSpot(TM) 64-Bit Server VM (build 25.74-b02, mixed mode)
> org.apache.samza.SamzaException: java.lang.UnsatisfiedLinkError:
> /home/myuser/path/apps-lib/libcql_engine12d.so: libcql_types_12d.so:
> cannot open shared object file: No such file or directory
> at org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:144)
> at
> org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:730)
> at
> org.apache.samza.container.SamzaContainer$.safeMain(SamzaCon
> tainer.scala:12
> 2)
> at
> org.apache.samza.container.SamzaContainer$.main(SamzaContainer.scala:89)
> at org.apache.samza.container.SamzaContainer.main(SamzaContaine
> r.scala)
>
>
> Also, I don¹t have problems calling the library from another non-samza
> application.
>
> Thanks,
> Angelica
>
>
> On 6/8/17, 6:25 PM, "Jagadish Venkatraman" <jagadish1...@gmail.com> wrote:
>
> >Hi Angelica,
> >
> >Linkage error usually occurs when the JVM runtime is not able to find the
> >actual path for the native library.
> >
> >I assume you encounter this error in your container when you are running
> >on
> >Yarn. Samza exposes a property called *task.opts* which will allow you to
> >pass in command line JVM options to your containers. I'd recommend setting
> >task.opts =
> >-Djava.library.path="D:/Projects/lib/your_library_dir/sub_dir/
> >
> > >> I am printing the value for java.library.path and I can see the
> >following path to the folder that contains the library cql_engine12d.
> >Are you printing it inside your StreamTask implemention?
> >
> >Also, Just to isolate that it's not a problem with your environment, you
> >can try invoking the library call from a non-Samza program?
> >
> >Let us know how it goes!
> >
> >Best,
> >Jagadish
> >
> >
> >On Thu, Jun 8, 2017 at 12:23 PM, Angelica Garcia-Gutierrez <
> >agar...@transcendinsights.com> wrote:
> >
> >> Hello,
> >> I am trying to rung a job that uses additional custom libraries
> >> (cql_engine12).
> >> During processing I can see the following error in one of the container
> >> logs:
> >>
> >>
> >> 2017-06-08 12:09:56 AsyncRunLoop [ERROR] Caught throwable and stopping
> >>run
> >> loop
> >> java.lang.UnsatisfiedLinkError: no cql_engine12d in java.library.path
> >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)
> >> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> >> at java.lang.System.loadLibrary(System.java:1122)
> >> at com.transcendinsights.cql.CqlEngine.<clinit>(CqlEngine.java:15)
> >> at samza.examples.wikipedia.task.MyTask.evaluate(MyTask.java:138)
> >> at samza.examples.wikipedia.task.MyTask.process(MyTask.java:118)
> >> at org.apache.samza.task.AsyncStreamTaskAdapter.process(
> >> AsyncStreamTaskAdapter.java:72)
> >> at org.apache.samza.task.AsyncStreamTaskAdapter.processAsync(
> >> AsyncStreamTaskAdapter.java:63)
> >> at org.apache.samza.container.TaskInstance$$anonfun$process$
> >> 1.apply$mcV$sp(TaskInstance.scala:162)
> >> at org.apache.samza.container.TaskInstanceExceptionHandler.
> >> maybeHandle(TaskInstanceExceptionHandler.scala:54)
> >> at org.apache.samza.container.TaskInstance.process(
> >> TaskInstance.scala:160)
> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
> >> process(AsyncRunLoop.java:425)
> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
> >> run(AsyncRunLoop.java:370)
> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
> >> access$300(AsyncRunLoop.java:311)
> >> at
> >>org.apache.samza.task.AsyncRunLoop.runTasks(AsyncRunLoop.java:225)
> >> at org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:154)
> >> at org.apache.samza.container.SamzaContainer.run(
> >> SamzaContainer.scala:730)
> >> at org.apache.samza.container.SamzaContainer$.safeMain(
> >> SamzaContainer.scala:122)
> >> at org.apache.samza.container.SamzaContainer$.main(
> >> SamzaContainer.scala:89)
> >> at org.apache.samza.container.SamzaContainer.main(
> >> SamzaContainer.scala)
> >> 2017-06-08 12:09:56 SamzaContainer [ERROR] Caught exception/error in
> >> process loop.
> >>
> >> I am printing the value for java.library.path and I can see the
> >>following
> >> path to the folder that contains the library cql_engine12d.
> >>
> >> Any advice about what other process can be overriding the value for my
> >> library path?
> >>
> >> Thanks,
> >> Angelica
> >>
> >>
> >>
> >>
> >>
> >> The information transmitted is intended only for the person or entity to
> >> which it is addressed
> >> and may contain CONFIDENTIAL material. If you receive this
> >> material/information in error,
> >> please contact the sender and delete or destroy the
> >>material/information.
> >
> >
> >
> >
> >--
> >Jagadish V,
> >Graduate Student,
> >Department of Computer Science,
> >Stanford University
>
> The information transmitted is intended only for the person or entity to
> which it is addressed
> and may contain CONFIDENTIAL material. If you receive this
> material/information in error,
> please contact the sender and delete or destroy the material/information.
>
--
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University
