Hi Jagadish,
I was able to verify the value for java.library.path in the init method.
It looks correct to me as includes the path to the directory where all the
dependent libraries are located.
Angelica
On 6/9/17, 11:01 AM, "Jagadish Venkatraman" <jagadish1...@gmail.com> wrote:
>Can you verify that the path that the JVM tries to find the library
>(present in the LinkageError) is valid and is a part of java.library.path
>?
>
>You can do this by printing `(System.getProperty("java.library.path"));`
>inside your StreamTask's *init* method.
>
>Best,
>Jagadish
>
>On Thu, Jun 8, 2017 at 9:22 PM, Angelica Garcia-Gutierrez <
>agar...@transcendinsights.com> wrote:
>
>> Thanks for the hint!
>> I made some progress.
>>
>> By adding task.opts to the job configuration file the JVM was able to
>>find
>> the library cql_engine12d. However, this library depends on some other
>> libraries e.g. cql_types_12d (located in the same directory as
>> cql_engine12d) and am hitting the following error:
>>
>> Java HotSpot(TM) 64-Bit Server VM (build 25.74-b02, mixed mode)
>> org.apache.samza.SamzaException: java.lang.UnsatisfiedLinkError:
>> /home/myuser/path/apps-lib/libcql_engine12d.so: libcql_types_12d.so:
>> cannot open shared object file: No such file or directory
>> at org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:144)
>> at
>> org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:730)
>> at
>> org.apache.samza.container.SamzaContainer$.safeMain(SamzaCon
>> tainer.scala:12
>> 2)
>> at
>> org.apache.samza.container.SamzaContainer$.main(SamzaContainer.scala:89)
>> at org.apache.samza.container.SamzaContainer.main(SamzaContaine
>> r.scala)
>>
>>
>> Also, I don¹t have problems calling the library from another non-samza
>> application.
>>
>> Thanks,
>> Angelica
>>
>>
>> On 6/8/17, 6:25 PM, "Jagadish Venkatraman" <jagadish1...@gmail.com>
>>wrote:
>>
>> >Hi Angelica,
>> >
>> >Linkage error usually occurs when the JVM runtime is not able to find
>>the
>> >actual path for the native library.
>> >
>> >I assume you encounter this error in your container when you are
>>running
>> >on
>> >Yarn. Samza exposes a property called *task.opts* which will allow you
>>to
>> >pass in command line JVM options to your containers. I'd recommend
>>setting
>> >task.opts =
>> >-Djava.library.path="D:/Projects/lib/your_library_dir/sub_dir/
>> >
>> > >> I am printing the value for java.library.path and I can see the
>> >following path to the folder that contains the library cql_engine12d.
>> >Are you printing it inside your StreamTask implemention?
>> >
>> >Also, Just to isolate that it's not a problem with your environment,
>>you
>> >can try invoking the library call from a non-Samza program?
>> >
>> >Let us know how it goes!
>> >
>> >Best,
>> >Jagadish
>> >
>> >
>> >On Thu, Jun 8, 2017 at 12:23 PM, Angelica Garcia-Gutierrez <
>> >agar...@transcendinsights.com> wrote:
>> >
>> >> Hello,
>> >> I am trying to rung a job that uses additional custom libraries
>> >> (cql_engine12).
>> >> During processing I can see the following error in one of the
>>container
>> >> logs:
>> >>
>> >>
>> >> 2017-06-08 12:09:56 AsyncRunLoop [ERROR] Caught throwable and
>>stopping
>> >>run
>> >> loop
>> >> java.lang.UnsatisfiedLinkError: no cql_engine12d in java.library.path
>> >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)
>> >> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
>> >> at java.lang.System.loadLibrary(System.java:1122)
>> >> at
>>com.transcendinsights.cql.CqlEngine.<clinit>(CqlEngine.java:15)
>> >> at samza.examples.wikipedia.task.MyTask.evaluate(MyTask.java:138)
>> >> at samza.examples.wikipedia.task.MyTask.process(MyTask.java:118)
>> >> at org.apache.samza.task.AsyncStreamTaskAdapter.process(
>> >> AsyncStreamTaskAdapter.java:72)
>> >> at org.apache.samza.task.AsyncStreamTaskAdapter.processAsync(
>> >> AsyncStreamTaskAdapter.java:63)
>> >> at org.apache.samza.container.TaskInstance$$anonfun$process$
>> >> 1.apply$mcV$sp(TaskInstance.scala:162)
>> >> at org.apache.samza.container.TaskInstanceExceptionHandler.
>> >> maybeHandle(TaskInstanceExceptionHandler.scala:54)
>> >> at org.apache.samza.container.TaskInstance.process(
>> >> TaskInstance.scala:160)
>> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
>> >> process(AsyncRunLoop.java:425)
>> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
>> >> run(AsyncRunLoop.java:370)
>> >> at org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.
>> >> access$300(AsyncRunLoop.java:311)
>> >> at
>> >>org.apache.samza.task.AsyncRunLoop.runTasks(AsyncRunLoop.java:225)
>> >> at org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:154)
>> >> at org.apache.samza.container.SamzaContainer.run(
>> >> SamzaContainer.scala:730)
>> >> at org.apache.samza.container.SamzaContainer$.safeMain(
>> >> SamzaContainer.scala:122)
>> >> at org.apache.samza.container.SamzaContainer$.main(
>> >> SamzaContainer.scala:89)
>> >> at org.apache.samza.container.SamzaContainer.main(
>> >> SamzaContainer.scala)
>> >> 2017-06-08 12:09:56 SamzaContainer [ERROR] Caught exception/error in
>> >> process loop.
>> >>
>> >> I am printing the value for java.library.path and I can see the
>> >>following
>> >> path to the folder that contains the library cql_engine12d.
>> >>
>> >> Any advice about what other process can be overriding the value for
>>my
>> >> library path?
>> >>
>> >> Thanks,
>> >> Angelica
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> The information transmitted is intended only for the person or
>>entity to
>> >> which it is addressed
>> >> and may contain CONFIDENTIAL material. If you receive this
>> >> material/information in error,
>> >> please contact the sender and delete or destroy the
>> >>material/information.
>> >
>> >
>> >
>> >
>> >--
>> >Jagadish V,
>> >Graduate Student,
>> >Department of Computer Science,
>> >Stanford University
>>
>> The information transmitted is intended only for the person or entity to
>> which it is addressed
>> and may contain CONFIDENTIAL material. If you receive this
>> material/information in error,
>> please contact the sender and delete or destroy the
>>material/information.
>>
>
>
>
>--
>Jagadish V,
>Graduate Student,
>Department of Computer Science,
>Stanford University
The information transmitted is intended only for the person or entity to which
it is addressed
and may contain CONFIDENTIAL material. If you receive this
material/information in error,
please contact the sender and delete or destroy the material/information.
