Hi Mike,
do you run Flink locally or in a cluster? You have to make sure that VM
argument -Djava.library.path is set for all Flink JVMs. Job Manager and
Task Managers might run in separate JVMs. Make also sure that the
library is accessible from all node. I don't know what happens if the
file is accessed by multiple processes/threads at the same time. It
might also important where you put the static { ... } loading. It should
be in the Function, because these classes get deserialized on the
TaskManager.
I hope this helps.
Timo
Am 17.07.17 um 21:30 schrieb Mike Accola:
I am new Flink user just trying to learn a little bit. I am trying to
incorporate an existing C++ library into a new Flink application. I am
seeing some strange behavior when trying to link in the native (C++)
library using java via JNI.
I am running this on Linux (RHEL6)
I can run my application once without error. Sometimes it will run
successfully a 2nd or 3rd time. However, eventually on a subsequent run,
I get an exception about the the native library not being found:
java.lang.UnsatisfiedLinkError: no dummy2native in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at com.att.flink.tdata.spss.TinyLoader.loadNative(Dummy2.java:10)
For debugging purposes for now, my native library does not have any
external references. It really contains 1 method that essentially does
nothing.
The behavior seems to indicate that there is some kind of cleanup being
done that "unloads" the native library. I suspect this is somehow related
to Flink's implementation of its library cache manager, but I have not
been able to prove this yet.
A few more details:
- I have a c++ library libdummy2native.so that contains a method that can
be invoked via JNI.
- I have a jar containing a class, called Dummy2. The Dummy2 constructor
will invoke the JNI method.
- The libdummy2native.so library is invoked with System.loadLibrary() like
this:
static {System.loadLibrary("dummy2native"); }
- In my simple Flink application, I have extended the ProcessFunction
class. Within this class, I have overriden processElement method that
declares a Dummy2 object.
- The Dummy2 class can be called and invoked without error when used in a
standalone java program.
Any thoughts or ideas on what to try next would be appreciated. Initially,
I'd be happy to be able to just explain this behavior. I will worry about
fixing it afterwards.
Thanks.
