On Thu, Jun 5, 2025 at 1:33 AM David Holmes <david.hol...@oracle.com> wrote: > > On 5/06/2025 1:33 am, Jiangli Zhou wrote: > > Ok, still thanks for the thoughts, David. > > > > To summarize, here is what we can do for the current step: > > > > - Allow JNI_OnLoad_L and etc for JDK internal native libraries with > > the dynamic linking case. > > - JNI spec already allows JNI_OnLoad_L and etc for dynamically linked > > native libraries. No spec change is required to the above. > > Sorry but where does the spec allow this? I thought we had agreed that > this was a limitation of the implementation in regard to detecting > whether a library is loaded dynamically or statically, and that we > needed to document this case in the spec. ??
It's based on the following (referenced in https://mail.openjdk.org/pipermail/leyden-dev/2025-May/002144.html message) specified in https://docs.oracle.com/en/java/javase/21/docs/specs/jni/invocation.html#support-for-statically-linked-libraries : Support for Statically Linked Libraries "If dynamically linked library defines JNI_OnLoad_L and/or JNI_OnUnload_L functions, these functions will be ignored." There are two parts to the above. The first part indicates that the spec allows JNI_OnLoad_L and/or JNI_OnUnload_L being defined in a dynamically linked library. The second part stating "these functions will be ignored" however does not reflect the limitation of the implementation when handling implicitly loaded native libraries. That part needs to be documented. Please let me know if that's more clear. Best, Jiangli > > David > ----- > > > - JNI_OnLoad_L and etc should not be called for dynamically linked > > native libraries when the library is loaded due to loadLibrary() by > > default. If a dynamic library is already loaded as a dependency of > > other native libraries, when loadLibrary() is called for the library, > > JNI_OnLoad_L can be called. That is an existing behavior since JDK 8. > > Need to document (in spec or release notes?). > > > > Best, > > Jiangli > > > > On Tue, Jun 3, 2025 at 9:31 PM David Holmes <david.hol...@oracle.com> wrote: > >> > >> On 4/06/2025 5:00 am, Jiangli Zhou wrote: > >>> On Mon, Jun 2, 2025 at 6:22 PM David Holmes <david.hol...@oracle.com> wrote: > >>>> > >>>> On 3/06/2025 9:29 am, Jiangli Zhou wrote: > >>>>> On Sun, Jun 1, 2025 at 7:55 PM David Holmes <david.hol...@oracle.com> wrote: > >>>>>> > >>>>>> On 31/05/2025 7:20 am, Jiangli Zhou wrote: > >>>>>>> On Thu, May 29, 2025 at 11:54 PM David Holmes < david.hol...@oracle.com> wrote: > >>>>>>>> > >>>>>>>> On 30/05/2025 9:26 am, Jiangli Zhou wrote: > >>>>>>>>> > >>>>>>>>> I just thought of one more thing related to the discussion now. Any > >>>>>>>>> concern if the implementation does not ignore JNI_OnLoad_L and etc if > >>>>>>>>> they are defined application's dynamically linked native libraries? Or > >>>>>>>>> that's unspecified behavior and it's up to the implement to decide? > >>>>>>>> > >>>>>>>> For Internal libraries or external? For external you have to follow the > >>>>>>>> spec - if both methods exist you only want to execute one of them. > >>>>>>> > >>>>>>> It's for the external (non-JDK) library that I'm a bit more cautious. > >>>>>>> > >>>>>>> In the existing code in JDK mainline, > >>>>>>> https://github.com/openjdk/jdk/blob/3cc630985d47be6ba4cf991698e999f17dbde203/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L117 , > >>>>>>> loadLibrary() first tries to find the built-in library using > >>>>>>> JNI_OnLoad_L symbol (L is the library name). When dlsym is called to > >>>>>>> find the symbol from the main process, any of the already loaded > >>>>>>> shared libraries are also searched, as described by the dlsym man page > >>>>>>> (included related part below). > >>>>>>> > >>>>>>> https://man7.org/linux/man-pages/man3/dlsym.3.html: > >>>>>>> RTLD_DEFAULT > >>>>>>> Find the first occurrence of the desired symbol using the > >>>>>>> default shared object search order. The search will > >>>>>>> include global symbols in the executable and its > >>>>>>> dependencies, as well as symbols in shared objects that > >>>>>>> were dynamically loaded with the RTLD_GLOBAL flag. > >>>>>>> > >>>>>>> I think it would be rare, it is possible to construct such case: > >>>>>>> > >>>>>>> There are user JNI libraries A and B, with B is built as a dependency > >>>>>>> of A. A defines JNI_OnLoad_A and JNI_OnLoad. B defines JNI_OnLoad_B > >>>>>>> and JNI_OnLoad. When A is being loaded using loadLibrary(), > >>>>>>> loadLibrary() tries first to lookup JNI_OnLoad_A, which is not found. > >>>>>>> A is then loaded dynamically, which causes B being loaded implicitly > >>>>>>> as a dependency of A. Later when loadLibrary() is called for B, > >>>>>>> JNI_OnLoad_B would be found and then called. This is an existing > >>>>>>> behavior. I think it's an unspecified behavior and we don't need to > >>>>>>> add any additional checks to prevent JNI_OnLoad_B from being called. > >>>>>> > >>>>>> That sounds like a significant design flaw to me. You can't specify that > >>>>>> JNI_OnLoad_L will only be called if L is statically linked, if the > >>>>>> existence of JNI_OnLoad_L is used to infer that L is statically linked! > >>>>>> I would expect libraries to have both versions of the OnLoad functions > >>>>>> to allow for them being statically or dynamically linked - which the > >>>>>> spec allows for by saying the alternate variant is ignored. But then the > >>>>>> JDK will execute the wrong method if it finds JNI_OnLoad_L in a > >>>>>> dynamically linked library. > >>>>> > >>>>> JNI_OnLoad_L is used to determine if a requested JNI native library is > >>>>> a built-in (statically linked) library. Thus, it can avoid the > >>>>> operation of explicit loading for the shared library, e.g. with dlopen > >>>>> on Linux. JNI_OnLoad_L is expected to provide the same implementation > >>>>> as JNI_OnLoad besides being used as an identifier of a built-in > >>>>> library, IIUC. > >>>>> > >>>>> In the scenario that I described in the previous message, when a JNI > >>>>> shared library is already implicitly loaded as a dependency of another > >>>>> native library, dlopen for explicitly loading the shared library is > >>>>> not necessary. From the implementation point of view, the code seems > >>>>> to have been doing the right thing since JDK 8. I did some search and > >>>>> found https://stackoverflow.com/questions/32302262/does-dlopen-re-load-already-loaded-dependencies-if-so-what-are-the-implication , > >>>>> which point out to the following in POSIX spec (latest > >>>>> https://pubs.opengroup.org/onlinepubs/9799919799/): > >>>>> > >>>>> "Only a single copy of an executable object file shall be brought into > >>>>> the address space, even if dlopen() is invoked multiple times in > >>>>> reference to the executable object file, and even if different > >>>>> pathnames are used to reference the executable object file." > >>>>> > >>>>> Then avoiding calling dlopen for the already loaded native library > >>>>> doesn't cause any undesired side effects. > >>>>> > >>>>> Perhaps we can update the JNI spec to include the "already loaded" JNI > >>>>> native libraries case, in addition to the built-in native libraries, > >>>>> regarding JNI_OnLoad_L. Also clarify the "these functions will be > >>>>> ignored" part in JNI spec for the dynamically linked libraries. > >>>>> > >>>>> "If dynamically linked library defines JNI_OnLoad_L and/or > >>>>> JNI_OnUnload_L functions, these functions will be ignored." > >>>>> > >>>>> Thoughts? > >>>> > >>>> The problem is, as I see it, that the spec assumes that if it finds the > >>>> JNI_OnLoad_L symbol then L must be a statically linked library. But that > >>>> ignores the case you highlight where L was implicitly dynamically loaded > >>>> as a dependency on another library. Hence the existence test for the > >>>> symbol is not sufficient to determine if L was statically linked. > >>> > >>> Right. > >>> > >>>> > >>>> If JNI_OnLoad_L and JNI_OnLoad were guaranteed to always do exactly the > >>>> same thing it would not make any practical difference, but surely that > >>>> is not always the case? I can certainly postulate the existence of a > >>>> library that only needs the "on load" hook for the statically linked > >>>> case, in which case invoking it when actually dynamically linked would > >>>> be incorrect and potentially harmful. > >>> > >>> Hmmm, I haven't thought of such a case. David, could you please give a > >>> concrete example for the case? > >> > >> I don't have a concrete example, that's why I just said I could > >> "postulate the existence" of such a case. :) > >> > >>>> > >>>> It seems we lack a way to know if a given library is truly statically > >>>> linked, or to be advised when a library is implicitly loaded as a > >>>> dependency. I am no expert on linking but I've been unable to locate any > >>>> information on programmatically determining these conditions. > >>> > >>> I haven't found anything either. > >>> > >>>> > >>>> If there is no real solution then documenting the problem may be all we > >>>> can do. > >>> > >>> That sounds reasonable to me. > >>> > >>> I mentioned the new mailing list discussions to Ron, Alan and Magnus > >>> today during the hermetic Java meeting. They may have follow up > >>> thoughts. > >> > >> Okay. > >> > >> David > >> ----- > >> > >> > >>> Thanks! > >>> Jiangli > >>> > >>>> > >>>> David > >>>> ----- > >>>> > >>>>> Thanks! > >>>>> Jiangli > >> >
