Apparently you're using System.load() to load the compiled helpers functions. It seems it may load the library using RTLD_LOCAL, which means it won't be seen from other dynamic libraries loaded in the same process.
According to this SO entry, the solution would be to use NativeLibrary.getInstance() instead: https://stackoverflow.com/questions/5425034/java-load-shared-librariees-with-dependencies Regards Antoine. Le 17/12/2018 à 16:48, Ravindra Pindikura a écrit : > > >> On Dec 17, 2018, at 6:39 PM, Antoine Pitrou <anto...@python.org> wrote: >> >> >> Is LoadLibraryPermanently(nullptr) called after the libgandiva_jni.so is >> loaded? Perhaps you can try LoadLibraryPermanently("libgandiva_jni.so") >> explicitly? > > We are already doing this inside gandiva. > > https://github.com/apache/arrow/blob/master/cpp/src/gandiva/engine.cc#L61 > <https://github.com/apache/arrow/blob/master/cpp/src/gandiva/engine.cc#L61> > >> >> >> Le 17/12/2018 à 13:50, Ravindra Pindikura a écrit : >>> Antoine, >>> >>> I set ARROW_GANDIVA_STATIC_LIBSTDCPP and modified the jni/CMakeLists.txt to >>> export all symbols (i.e removed the version-script parameter in LINK_FLAGS). >>> >>> $ nm ./debug/libgandiva_jni.so|grep _ZNSt8ios_base4InitC1Ev >>> 0000000001df34f0 T _ZNSt8ios_base4InitC1Ev >>> >>> $ nm --defined-only ./debug/libgandiva_jni.so|grep _ZNSt8ios_base4InitC1Ev >>> 0000000001df34f0 T _ZNSt8ios_base4InitC1Ev >>> >>> I still get the same “could not be resolved!” Error when I run the java >>> test. It appears like when the library is loaded via jni, some symbols are >>> getting hidden from the JIT. >>> >>> Of the four sub-cases of the test : >>> >>> 1. a cpp executable (shared/static), the decimal functions invoked from CPP >>> fns >>> 2. a cpp executable (shared/static), the decimal functions invoked from JIT >>> code >>> 3. a java executable, the decimal functions invoked from CPP Fns >>> 4. a java executable, the decimal functions invoked from JIT code >>> >>> 1, 2 & 3 work good. Only 4 causes the symbol resolution issue. >>> >>> Thanks & regards, >>> Ravindra. >>> >>> >>>> On Dec 16, 2018, at 6:34 PM, Antoine Pitrou <anto...@python.org> wrote: >>>> >>>> >>>> In other words, see if: >>>> nm --defined-only libgandiva.a | grep _ZNSt8ios_base4InitC1Ev >>>> >>>> outputs anything. >>>> >>>> Regards >>>> >>>> Antoine. >>>> >>>> >>>> Le 16/12/2018 à 14:03, Antoine Pitrou a écrit : >>>>> >>>>> Does the static gandiva lib bundle libstdc++? Otherwise you'll probably >>>>> need to load it explicitly in the Java process. >>>>> >>>>> Regards >>>>> >>>>> Antoine. >>>>> >>>>> >>>>> Le 16/12/2018 à 04:53, Ravindra Pindikura a écrit : >>>>>> I think this happens if any of the cpp files that are used for IR >>>>>> generation (clang —emit-llvm) includes <iostream>. For decimal support, >>>>>> I’m generating IR code from arrow/util/decimal.cc <http://decimal.cc/> >>>>>> which includes <iostream> (many times in the header file chain). >>>>>> >>>>>>> On Dec 16, 2018, at 9:14 AM, Ravindra Pindikura <ravin...@dremio.com> >>>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I’ve been working on adding decimal support in gandiva - I’m hitting a >>>>>>> symbol resolution error with std::ios_base::Init with the java tests on >>>>>>> ubuntu. Any ideas on what I may be doing wrong ? >>>>>>> >>>>>>> LLVM ERROR: Program used external function '_ZNSt8ios_base4InitC1Ev' >>>>>>> which could not be resolved! >>>>>>> >>>>>>> >>>>>>> I’ve the same test (add two decimals) in three forms : >>>>>>> >>>>>>> 1. Cpp test executable with shared gandiva lib >>>>>>> >>>>>>> Works on both ubuntu and Mac >>>>>>> >>>>>>> 2. Cpp test executable with static gandiva lib >>>>>>> >>>>>>> Works on both ubuntu and Mac >>>>>>> >>>>>>> 3. Java test (internally uses jni over static gandiva lib) >>>>>>> >>>>>>> Works on Mac, but fails on ubuntu !! >>>>>>> >>>>>>> Ubuntu failure : https://travis-ci.org/apache/arrow/jobs/468562791 >>>>>>> >>>>>>> I tried switching to ORC JIT (from MC JIT) but that didn’t help. In the >>>>>>> JIT, the symbol is not found in the process address space for >>>>>>> ubuntu+java. For all the other cases, the symbol is found. >>>>>>> >>>>>>> https://github.com/apache/arrow/pull/2942/files#diff-5ef29a5cabe384645dbf7a9c704aceeaR36 >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Ravindra. >>>>>> >>>>>> >>> > >
