On 5/06/2025 1:33 am, Jiangli Zhou wrote:
Ok, still thanks for the thoughts, David.
To summarize, here is what we can do for the current step:
- Allow JNI_OnLoad_L and etc for JDK internal native libraries with
the dynamic linking case.
- JNI spec already allows JNI_OnLoad_L and etc for dynamically linked
native libraries. No spec change is required to the above.
Sorry but where does the spec allow this? I thought we had agreed that
this was a limitation of the implementation in regard to detecting
whether a library is loaded dynamically or statically, and that we
needed to document this case in the spec. ??
David
-----
- JNI_OnLoad_L and etc should not be called for dynamically linked
native libraries when the library is loaded due to loadLibrary() by
default. If a dynamic library is already loaded as a dependency of
other native libraries, when loadLibrary() is called for the library,
JNI_OnLoad_L can be called. That is an existing behavior since JDK 8.
Need to document (in spec or release notes?).
Best,
Jiangli
On Tue, Jun 3, 2025 at 9:31 PM David Holmes <david.hol...@oracle.com> wrote:
On 4/06/2025 5:00 am, Jiangli Zhou wrote:
On Mon, Jun 2, 2025 at 6:22 PM David Holmes <david.hol...@oracle.com> wrote:
On 3/06/2025 9:29 am, Jiangli Zhou wrote:
On Sun, Jun 1, 2025 at 7:55 PM David Holmes <david.hol...@oracle.com> wrote:
On 31/05/2025 7:20 am, Jiangli Zhou wrote:
On Thu, May 29, 2025 at 11:54 PM David Holmes <david.hol...@oracle.com> wrote:
On 30/05/2025 9:26 am, Jiangli Zhou wrote:
I just thought of one more thing related to the discussion now. Any
concern if the implementation does not ignore JNI_OnLoad_L and etc if
they are defined application's dynamically linked native libraries? Or
that's unspecified behavior and it's up to the implement to decide?
For Internal libraries or external? For external you have to follow the
spec - if both methods exist you only want to execute one of them.
It's for the external (non-JDK) library that I'm a bit more cautious.
In the existing code in JDK mainline,
https://github.com/openjdk/jdk/blob/3cc630985d47be6ba4cf991698e999f17dbde203/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L117,
loadLibrary() first tries to find the built-in library using
JNI_OnLoad_L symbol (L is the library name). When dlsym is called to
find the symbol from the main process, any of the already loaded
shared libraries are also searched, as described by the dlsym man page
(included related part below).
https://man7.org/linux/man-pages/man3/dlsym.3.html:
RTLD_DEFAULT
Find the first occurrence of the desired symbol using the
default shared object search order. The search will
include global symbols in the executable and its
dependencies, as well as symbols in shared objects that
were dynamically loaded with the RTLD_GLOBAL flag.
I think it would be rare, it is possible to construct such case:
There are user JNI libraries A and B, with B is built as a dependency
of A. A defines JNI_OnLoad_A and JNI_OnLoad. B defines JNI_OnLoad_B
and JNI_OnLoad. When A is being loaded using loadLibrary(),
loadLibrary() tries first to lookup JNI_OnLoad_A, which is not found.
A is then loaded dynamically, which causes B being loaded implicitly
as a dependency of A. Later when loadLibrary() is called for B,
JNI_OnLoad_B would be found and then called. This is an existing
behavior. I think it's an unspecified behavior and we don't need to
add any additional checks to prevent JNI_OnLoad_B from being called.
That sounds like a significant design flaw to me. You can't specify that
JNI_OnLoad_L will only be called if L is statically linked, if the
existence of JNI_OnLoad_L is used to infer that L is statically linked!
I would expect libraries to have both versions of the OnLoad functions
to allow for them being statically or dynamically linked - which the
spec allows for by saying the alternate variant is ignored. But then the
JDK will execute the wrong method if it finds JNI_OnLoad_L in a
dynamically linked library.
JNI_OnLoad_L is used to determine if a requested JNI native library is
a built-in (statically linked) library. Thus, it can avoid the
operation of explicit loading for the shared library, e.g. with dlopen
on Linux. JNI_OnLoad_L is expected to provide the same implementation
as JNI_OnLoad besides being used as an identifier of a built-in
library, IIUC.
In the scenario that I described in the previous message, when a JNI
shared library is already implicitly loaded as a dependency of another
native library, dlopen for explicitly loading the shared library is
not necessary. From the implementation point of view, the code seems
to have been doing the right thing since JDK 8. I did some search and
found
https://stackoverflow.com/questions/32302262/does-dlopen-re-load-already-loaded-dependencies-if-so-what-are-the-implication,
which point out to the following in POSIX spec (latest
https://pubs.opengroup.org/onlinepubs/9799919799/):
"Only a single copy of an executable object file shall be brought into
the address space, even if dlopen() is invoked multiple times in
reference to the executable object file, and even if different
pathnames are used to reference the executable object file."
Then avoiding calling dlopen for the already loaded native library
doesn't cause any undesired side effects.
Perhaps we can update the JNI spec to include the "already loaded" JNI
native libraries case, in addition to the built-in native libraries,
regarding JNI_OnLoad_L. Also clarify the "these functions will be
ignored" part in JNI spec for the dynamically linked libraries.
"If dynamically linked library defines JNI_OnLoad_L and/or
JNI_OnUnload_L functions, these functions will be ignored."
Thoughts?
The problem is, as I see it, that the spec assumes that if it finds the
JNI_OnLoad_L symbol then L must be a statically linked library. But that
ignores the case you highlight where L was implicitly dynamically loaded
as a dependency on another library. Hence the existence test for the
symbol is not sufficient to determine if L was statically linked.
Right.
If JNI_OnLoad_L and JNI_OnLoad were guaranteed to always do exactly the
same thing it would not make any practical difference, but surely that
is not always the case? I can certainly postulate the existence of a
library that only needs the "on load" hook for the statically linked
case, in which case invoking it when actually dynamically linked would
be incorrect and potentially harmful.
Hmmm, I haven't thought of such a case. David, could you please give a
concrete example for the case?
I don't have a concrete example, that's why I just said I could
"postulate the existence" of such a case. :)
It seems we lack a way to know if a given library is truly statically
linked, or to be advised when a library is implicitly loaded as a
dependency. I am no expert on linking but I've been unable to locate any
information on programmatically determining these conditions.
I haven't found anything either.
If there is no real solution then documenting the problem may be all we
can do.
That sounds reasonable to me.
I mentioned the new mailing list discussions to Ron, Alan and Magnus
today during the hermetic Java meeting. They may have follow up
thoughts.
Okay.
David
-----
Thanks!
Jiangli
David
-----
Thanks!
Jiangli
