On Sun, Jun 8, 2025 at 8:19 PM David Holmes <david.hol...@oracle.com> wrote:
> On 6/06/2025 1:23 am, Jiangli Zhou wrote: > > On Thu, Jun 5, 2025 at 1:33 AM David Holmes <david.hol...@oracle.com > > <mailto:david.hol...@oracle.com>> wrote: > > > > > > On 5/06/2025 1:33 am, Jiangli Zhou wrote: > > > > Ok, still thanks for the thoughts, David. > > > > > > > > To summarize, here is what we can do for the current step: > > > > > > > > - Allow JNI_OnLoad_L and etc for JDK internal native libraries with > > > > the dynamic linking case. > > > > - JNI spec already allows JNI_OnLoad_L and etc for dynamically > linked > > > > native libraries. No spec change is required to the above. > > > > > > Sorry but where does the spec allow this? I thought we had agreed that > > > this was a limitation of the implementation in regard to detecting > > > whether a library is loaded dynamically or statically, and that we > > > needed to document this case in the spec. ?? > > > > It's based on the following (referenced in https://mail.openjdk.org/ > > pipermail/leyden-dev/2025-May/002144.html <https://mail.openjdk.org/ > > pipermail/leyden-dev/2025-May/002144.html> message) specified in > > https://docs.oracle.com/en/java/javase/21/docs/specs/jni/ > > invocation.html#support-for-statically-linked-libraries <https:// > > docs.oracle.com/en/java/javase/21/docs/specs/jni/ > > invocation.html#support-for-statically-linked-libraries>: > > > > Support for Statically Linked Libraries > > > > > > "If dynamically linked library defines JNI_OnLoad_L and/or > > JNI_OnUnload_L functions, these functions will be ignored." > > > > There are two parts to the above. The first part indicates that the spec > > allows JNI_OnLoad_L and/or JNI_OnUnload_L being defined in a dynamically > > linked library. > > I don't agree that you can split that one sentence into two parts to > allow the first section to be a justification of allowing JNI_OnLoad_L > in a dynamically linked library. The intent of this part of to say these > functions should not exist but if they do we will ignore them. > > We have a situation where the spec is clear in intent, but the > implementation doesn't have enough information to accurately implement > what the spec states. > > So the intent here is to document this gap between the spec and > implementation by adding a note to the spec. > Thanks, David. Your interpretation and reasoning sounds ok to me. More importantly, we have consensus on documenting the case with implicitly loaded dynamically linked native libraries. I'll file a bug. Best Jiangli > > David > ----- > > > The second part stating "these functions will be ignored" however does > > not reflect the limitation of the implementation when handling > > implicitly loaded native libraries. That part needs to be documented. > > > > Please let me know if that's more clear. > > > > Best, > > > > Jiangli > > > > > > > > David > > > ----- > > > > > > > - JNI_OnLoad_L and etc should not be called for dynamically linked > > > > native libraries when the library is loaded due to loadLibrary() by > > > > default. If a dynamic library is already loaded as a dependency of > > > > other native libraries, when loadLibrary() is called for the > library, > > > > JNI_OnLoad_L can be called. That is an existing behavior since JDK > 8. > > > > Need to document (in spec or release notes?). > > > > > > > > Best, > > > > Jiangli > > > > > > > > On Tue, Jun 3, 2025 at 9:31 PM David Holmes > > <david.hol...@oracle.com <mailto:david.hol...@oracle.com>> wrote: > > > >> > > > >> On 4/06/2025 5:00 am, Jiangli Zhou wrote: > > > >>> On Mon, Jun 2, 2025 at 6:22 PM David Holmes > > <david.hol...@oracle.com <mailto:david.hol...@oracle.com>> wrote: > > > >>>> > > > >>>> On 3/06/2025 9:29 am, Jiangli Zhou wrote: > > > >>>>> On Sun, Jun 1, 2025 at 7:55 PM David Holmes > > <david.hol...@oracle.com <mailto:david.hol...@oracle.com>> wrote: > > > >>>>>> > > > >>>>>> On 31/05/2025 7:20 am, Jiangli Zhou wrote: > > > >>>>>>> On Thu, May 29, 2025 at 11:54 PM David Holmes > > <david.hol...@oracle.com <mailto:david.hol...@oracle.com>> wrote: > > > >>>>>>>> > > > >>>>>>>> On 30/05/2025 9:26 am, Jiangli Zhou wrote: > > > >>>>>>>>> > > > >>>>>>>>> I just thought of one more thing related to the discussion > > now. Any > > > >>>>>>>>> concern if the implementation does not ignore JNI_OnLoad_L > > and etc if > > > >>>>>>>>> they are defined application's dynamically linked native > > libraries? Or > > > >>>>>>>>> that's unspecified behavior and it's up to the implement to > > decide? > > > >>>>>>>> > > > >>>>>>>> For Internal libraries or external? For external you have to > > follow the > > > >>>>>>>> spec - if both methods exist you only want to execute one of > > them. > > > >>>>>>> > > > >>>>>>> It's for the external (non-JDK) library that I'm a bit more > > cautious. > > > >>>>>>> > > > >>>>>>> In the existing code in JDK mainline, > > > >>>>>>> https://github.com/openjdk/jdk/ > > blob/3cc630985d47be6ba4cf991698e999f17dbde203/src/java.base/share/ > > classes/jdk/internal/loader/NativeLibraries.java#L117 <https:// > > github.com/openjdk/jdk/blob/3cc630985d47be6ba4cf991698e999f17dbde203/ > > > src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L117>, > > > >>>>>>> loadLibrary() first tries to find the built-in library using > > > >>>>>>> JNI_OnLoad_L symbol (L is the library name). When dlsym is > > called to > > > >>>>>>> find the symbol from the main process, any of the already > loaded > > > >>>>>>> shared libraries are also searched, as described by the dlsym > > man page > > > >>>>>>> (included related part below). > > > >>>>>>> > > > >>>>>>> https://man7.org/linux/man-pages/man3/dlsym.3.html <https:// > > man7.org/linux/man-pages/man3/dlsym.3.html>: > > > >>>>>>> RTLD_DEFAULT > > > >>>>>>> Find the first occurrence of the desired > > symbol using the > > > >>>>>>> default shared object search order. The > > search will > > > >>>>>>> include global symbols in the executable > > and its > > > >>>>>>> dependencies, as well as symbols in shared > > objects that > > > >>>>>>> were dynamically loaded with the > > RTLD_GLOBAL flag. > > > >>>>>>> > > > >>>>>>> I think it would be rare, it is possible to construct such > case: > > > >>>>>>> > > > >>>>>>> There are user JNI libraries A and B, with B is built as a > > dependency > > > >>>>>>> of A. A defines JNI_OnLoad_A and JNI_OnLoad. B defines > > JNI_OnLoad_B > > > >>>>>>> and JNI_OnLoad. When A is being loaded using loadLibrary(), > > > >>>>>>> loadLibrary() tries first to lookup JNI_OnLoad_A, which is > > not found. > > > >>>>>>> A is then loaded dynamically, which causes B being loaded > > implicitly > > > >>>>>>> as a dependency of A. Later when loadLibrary() is called for > B, > > > >>>>>>> JNI_OnLoad_B would be found and then called. This is an > existing > > > >>>>>>> behavior. I think it's an unspecified behavior and we don't > > need to > > > >>>>>>> add any additional checks to prevent JNI_OnLoad_B from being > > called. > > > >>>>>> > > > >>>>>> That sounds like a significant design flaw to me. You can't > > specify that > > > >>>>>> JNI_OnLoad_L will only be called if L is statically linked, if > the > > > >>>>>> existence of JNI_OnLoad_L is used to infer that L is > > statically linked! > > > >>>>>> I would expect libraries to have both versions of the OnLoad > > functions > > > >>>>>> to allow for them being statically or dynamically linked - > > which the > > > >>>>>> spec allows for by saying the alternate variant is ignored. > > But then the > > > >>>>>> JDK will execute the wrong method if it finds JNI_OnLoad_L in a > > > >>>>>> dynamically linked library. > > > >>>>> > > > >>>>> JNI_OnLoad_L is used to determine if a requested JNI native > > library is > > > >>>>> a built-in (statically linked) library. Thus, it can avoid the > > > >>>>> operation of explicit loading for the shared library, e.g. with > > dlopen > > > >>>>> on Linux. JNI_OnLoad_L is expected to provide the same > > implementation > > > >>>>> as JNI_OnLoad besides being used as an identifier of a built-in > > > >>>>> library, IIUC. > > > >>>>> > > > >>>>> In the scenario that I described in the previous message, when > > a JNI > > > >>>>> shared library is already implicitly loaded as a dependency of > > another > > > >>>>> native library, dlopen for explicitly loading the shared > library is > > > >>>>> not necessary. From the implementation point of view, the code > > seems > > > >>>>> to have been doing the right thing since JDK 8. I did some > > search and > > > >>>>> found https://stackoverflow.com/questions/32302262/does-dlopen- > > re-load-already-loaded-dependencies-if-so-what-are-the-implication > > <https://stackoverflow.com/questions/32302262/does-dlopen-re-load- > > already-loaded-dependencies-if-so-what-are-the-implication>, > > > >>>>> which point out to the following in POSIX spec (latest > > > >>>>> https://pubs.opengroup.org/onlinepubs/9799919799/ <https:// > > pubs.opengroup.org/onlinepubs/9799919799/>): > > > >>>>> > > > >>>>> "Only a single copy of an executable object file shall be > > brought into > > > >>>>> the address space, even if dlopen() is invoked multiple times in > > > >>>>> reference to the executable object file, and even if different > > > >>>>> pathnames are used to reference the executable object file." > > > >>>>> > > > >>>>> Then avoiding calling dlopen for the already loaded native > library > > > >>>>> doesn't cause any undesired side effects. > > > >>>>> > > > >>>>> Perhaps we can update the JNI spec to include the "already > > loaded" JNI > > > >>>>> native libraries case, in addition to the built-in native > > libraries, > > > >>>>> regarding JNI_OnLoad_L. Also clarify the "these functions will > be > > > >>>>> ignored" part in JNI spec for the dynamically linked libraries. > > > >>>>> > > > >>>>> "If dynamically linked library defines JNI_OnLoad_L and/or > > > >>>>> JNI_OnUnload_L functions, these functions will be ignored." > > > >>>>> > > > >>>>> Thoughts? > > > >>>> > > > >>>> The problem is, as I see it, that the spec assumes that if it > > finds the > > > >>>> JNI_OnLoad_L symbol then L must be a statically linked library. > > But that > > > >>>> ignores the case you highlight where L was implicitly > > dynamically loaded > > > >>>> as a dependency on another library. Hence the existence test for > the > > > >>>> symbol is not sufficient to determine if L was statically linked. > > > >>> > > > >>> Right. > > > >>> > > > >>>> > > > >>>> If JNI_OnLoad_L and JNI_OnLoad were guaranteed to always do > > exactly the > > > >>>> same thing it would not make any practical difference, but > > surely that > > > >>>> is not always the case? I can certainly postulate the existence > of a > > > >>>> library that only needs the "on load" hook for the statically > linked > > > >>>> case, in which case invoking it when actually dynamically linked > > would > > > >>>> be incorrect and potentially harmful. > > > >>> > > > >>> Hmmm, I haven't thought of such a case. David, could you please > > give a > > > >>> concrete example for the case? > > > >> > > > >> I don't have a concrete example, that's why I just said I could > > > >> "postulate the existence" of such a case. :) > > > >> > > > >>>> > > > >>>> It seems we lack a way to know if a given library is truly > > statically > > > >>>> linked, or to be advised when a library is implicitly loaded as a > > > >>>> dependency. I am no expert on linking but I've been unable to > > locate any > > > >>>> information on programmatically determining these conditions. > > > >>> > > > >>> I haven't found anything either. > > > >>> > > > >>>> > > > >>>> If there is no real solution then documenting the problem may be > > all we > > > >>>> can do. > > > >>> > > > >>> That sounds reasonable to me. > > > >>> > > > >>> I mentioned the new mailing list discussions to Ron, Alan and > Magnus > > > >>> today during the hermetic Java meeting. They may have follow up > > > >>> thoughts. > > > >> > > > >> Okay. > > > >> > > > >> David > > > >> ----- > > > >> > > > >> > > > >>> Thanks! > > > >>> Jiangli > > > >>> > > > >>>> > > > >>>> David > > > >>>> ----- > > > >>>> > > > >>>>> Thanks! > > > >>>>> Jiangli > > > >> > > > > >
