Hi Johan, Thanks as always for the explanation, and some of the history on this. I'll go the jmods route to avoid this. Agreed a warning on this *somewhere* would be good, as it's not something developers typically need to consider.
Kind Regards, Cormac On Sun, 9 Feb 2025 at 10:56, Johan Vos <johan....@gluonhq.com> wrote: > Hi Cormac, > > I understand the problem, and I agree it can be really annoying. It either > needs better communication, or a fix. The maven artifacts are something we > added with Gluon (hence not Oracle). As you said, windows signing > certificates come with a price (and building hassle). > The reason we started uploading those artifacts is mainly because > developers are very used to the maven artifact approach during development. > I believe this lowered the bar for JavaFX development after the JDK didn't > ship with the JavaFX code anymore. A developer who just wants to explore > using JavaFX is less likely to download an additional SDK or jmods. Simply > adding a few lines in a pom.xml is something most developers are familiar > with. > For shipping applications, I strongly discourage this, and > highly recommend the SDK/jmods approach. I can see now though that offering > the "maven artifacts for developers" created higher expectations than > intended. > > The problem you described is more or less documented in > https://bugs.openjdk.org/browse/JDK-8316276 . There have been discussions > (e.g. on the jigsaw list) in the past about this, as other projects (e.g. > dl4j) have similar issues with jars containing native code. There is no > standard approach in Java to deal with this, which is why the > NativeLibLoader contains a bunch of logic that allows to invoke the code in > the native libs from the classes in the same jar. That logic has been > modified over the years, to account for specific issues (e.g. [1] and [2]), > but it's not perfect. > > I'm not against signing those libraries (it is a net loss of money and > time though), but as you noticed yourself, this won't fix all problems. We > should probably make it more clear that the maven artifacts should not be > used in production systems, and are for development only -- for developers > by developers. > > - Johan > > [1] https://bugs.openjdk.org/browse/JDK-8317308 > [2] https://bugs.openjdk.org/browse/JDK-8307536 > > On Sat, Feb 8, 2025 at 10:01 PM Cormac Redmond <credm...@certak.com> > wrote: > >> Hi Steve, >> >> Thanks. I can also workaround it (with very limited changes) in another >> way by overriding javafx.runtime.version with something specific to my >> app, and its version (which will ultimately dictate the cache directory >> name) -- as the chances of having a clash then, are astronomically low. >> >> However, none of these things are solutions, but workarounds -- my stance >> is that it should not be up to the developer to firstly know this problem >> even exists (most won't), and then to workaround it by avoiding using >> certain JARs. It's not like we're building apps in un-documented or unusual >> ways, quite the opposite. >> >> >> Kind Regards, >> Cormac >> >> On Sat, 8 Feb 2025 at 18:16, Steve Hannah <st...@weblite.ca> wrote: >> >>> This also doesn't seem to affect apps running with the zulufx >>> distributions (the JRE with JavaFX bundled). >>> >>> E.g. this build of JavaFX ensemble >>> https://www.jdeploy.com/~jdeploy-demo-javafx-ensemble >>> It uses Java 19, and OpenJFX 20, and it doesn't seem to create an >>> .openjfx directory anywhere that I can find. >>> >>> Therefore, you can deploy your app using jDeploy and it won't have this >>> issue. (Disclosure, I'm the creator of jDeploy). Similarly, if you bundle >>> your app with a ZuluFX distribution, and strip out your maven jars (which >>> is what jDeploy does), you won't have this issue. >>> >>> Steve >>> >>> On Sat, Feb 8, 2025 at 9:35 AM Cormac Redmond <credm...@certak.com> >>> wrote: >>> >>>> Hi, >>>> >>>> Thanks for the reply. Yes -- my project uses JFX JARs rather than jmods >>>> (as many do). And the clashing application in question, must be the same. >>>> But this is a real problem that occurred with a real user, and I only have >>>> a handful of users. >>>> >>>> Oracle signing the JFX DLLs, while an improvement, would still leave >>>> the following problems wide open: >>>> >>>> - There's no way to stop a developer (or some build tool) from >>>> re-signing or removing signature from signed DLLs, in which case this >>>> problem can still re-occur >>>> - In the event of a genuine difference of a DLLs (under the same >>>> cache folder version), if the DLL cannot be deleted (to be replaced), >>>> because a running application application is using it -- a completely >>>> feasibly scenario -- then we have one application breaking another. >>>> >>>> Also, the developers shipping apps are in full control of the JFX JARs, >>>> their DLLs, and the *reported" javafx.version and javafx.runtime.version. >>>> E.g., I could have JFX 21.0.5 in a JFX 23.0.2 cache folder if I wanted >>>> simply by changing javafx.runtime.version (or javafx.cachedir). >>>> >>>> It's far too brittle. >>>> >>>> >>>> >>>> Regards, >>>> >>>> *Cormac Redmond* >>>> Software Engineer, Certak Ltd. >>>> >>>> e: credm...@certak.com | m: +353 (0) 86 268 2152 | w: www.certak.com >>>> >>>> >>>> >>>> >>>> On Sat, 8 Feb 2025 at 12:49, Christopher Schnick <crschn...@xpipe.io> >>>> wrote: >>>> >>>>> I think that went a bit under the radar as this only occurs when using >>>>> the maven dependencies. The SDK and jmod downloads do not have that issue >>>>> as they don't copy DLLs into any temp directory. Only the maven jars have >>>>> to do this. >>>>> >>>>> About signing any JavaFX DLLs, that would indeed be a good addition >>>>> considering all other JDK DLLs are signed. >>>>> >>>>> Best >>>>> Christopher Schnick >>>>> On 08/02/2025 13:31, Cormac Redmond wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am surprised nobody else sees this bug as a higher-priority >>>>> conversation point. >>>>> >>>>> It's troubling to see how running one self-contained application can >>>>> break another self-contained application (because of a cache that most JFX >>>>> devs wouldn't even know exist). >>>>> >>>>> If one well-behaved JFX application cannot delete/replace a file JFX >>>>> cache on start-up, because another well-behaved JFX application is using >>>>> that cached file (it will be if built on the same JFX version) -- then the >>>>> application will not run, at all. >>>>> >>>>> And as I explained earlier, this is a likely occurring scenario in the >>>>> wild -- the only reason this bug isn't more prevalent /reported / >>>>> noticeable, is that it's not too likely for a user to have two JFX >>>>> applications using the same JavaFX version, on their machine. But that's >>>>> down to pure coincidence / chance. I wouldn't say the same for Electron or >>>>> Flutter, etc. If they had this bug, it would be noticed and it would be >>>>> news. >>>>> >>>>> Also, the creators of these applications would have no idea that their >>>>> application is not starting, nor would the users know why. >>>>> >>>>> By the way, the problem is not just about *signed* DLLs + checksums, >>>>> obviously. It would occur if the DLLs are different for any other >>>>> reason too, obviously, and the authors of NativeLibLoader thought this >>>>> possibility is high enough to do the checksum + delete + replace check. >>>>> The >>>>> application shouldn't silently fail because of a cache management bug. >>>>> >>>>> >>>>> >>>>> >>>>> Regards, >>>>> >>>>> *Cormac Redmond* >>>>> Software Engineer, Certak Ltd. >>>>> >>>>> e: credm...@certak.com | m: +353 (0) 86 268 2152 | w: www.certak.com >>>>> >>>>> >>>>> >>>>> On Thu, 6 Feb 2025 at 19:56, Cormac Redmond <credm...@certak.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have found a "serious" bug, where two completely independent JFX >>>>>> applications, both with their own embedded runtime (built with jlink & >>>>>> jpackage) & using the same JavaFX version, are unable to run >>>>>> simultaneously, because of the JFX cache -- at least on Windows. >>>>>> >>>>>> When trying to run any application, NativeLibLoader does a checksum >>>>>> on DLLs in the cache (e.g.: >>>>>> C:\Users\xyz\.openjfx\cache\23.0.2+3\amd64\prism_d3d.dll); >>>>>> and tries to delete any files that exist where the checksums do not match >>>>>> (in order to replace them): >>>>>> >>>>>> if (!Arrays.equals(isHash, fileHash)) { >>>>>>> Files.delete(f.toPath()); >>>>>>> } >>>>>> >>>>>> >>>>>> But a second application *fails* to start, as it is unable to delete >>>>>> these files because they're in use by the first application (see >>>>>> stacktrace >>>>>> below). >>>>>> >>>>>> But why are the checksums different, shouldn't they be the same? They >>>>>> are different because the DLLs are signed by the builder of the >>>>>> applications -- different authors and different timestamps. So the DLL >>>>>> checksums will be different despite the DLLs being the same, in terms of >>>>>> the JFX version. >>>>>> >>>>>> Why are these JFX DLLs signed by the authors? Because for some >>>>>> reason, they come *unsigned*, and all DLLs when packaged *should* be >>>>>> signed, including any embedded in JARs, to avoid alarming Windows >>>>>> Defender >>>>>> warnings and mistrust, etc. There's no point spending a small fortune on >>>>>> Windows code-signing certs and shipping with any unsigned third-party >>>>>> DLLs >>>>>> (including any embedded in JARs). You might sign your own binaries and >>>>>> any >>>>>> unsigned third-party binaries. Similarly for MacOS, everything, including >>>>>> embedded native libs in JARs, etc., needs to be signed for gatekeeper & >>>>>> notarization to allow it. >>>>>> >>>>>> So it would be better if JFX DLLs came signed, to avoid forcing >>>>>> developers to do it (which would also avoid this checksum mishap). Or, if >>>>>> developers need to sign Oracle's DLLs, then this checksum approach isn't >>>>>> suitable. Also, there should really be a proper fallback in place -- a >>>>>> cache bug should not have such a catastrophic outcome. >>>>>> >>>>>> FYI: JLink also removes signatures from a bunch of JDK DLLs when >>>>>> assembling the runtime, but leaves a bunch of Windows DLLs >>>>>> untouched. Personally, I'd prefer all DLLs to come signed, and let the >>>>>> developers re-sign if they want. Removing the signatures is adding >>>>>> unnecessary hurdles for folks, for no benefit I can see. >>>>>> >>>>>> Anyway, example stacktrace: >>>>>> >>>>>> Loading D3D native library ... >>>>>>> WARNING: java.lang.UnsatisfiedLinkError: Can't load library: >>>>>>> C:\Program Files\KafkIO\bin\prism_d3d.dll >>>>>>> Loading library prism_d3d from resource failed: >>>>>>> java.nio.file.AccessDeniedException: C:\Users\xyz >>>>>>> \.openjfx\cache\23.0.2+3\amd64\prism_d3d.dll >>>>>>> java.nio.file.AccessDeniedException: C:\Users\xyz >>>>>>> \.openjfx\cache\23.0.2+3\amd64\prism_d3d.dll >>>>>>> at >>>>>>> java.base/sun.nio.fs.WindowsException.translateToIOException(Unknown >>>>>>> Source) >>>>>>> at >>>>>>> java.base/sun.nio.fs.WindowsException.rethrowAsIOException(Unknown >>>>>>> Source) >>>>>>> at >>>>>>> java.base/sun.nio.fs.WindowsException.rethrowAsIOException(Unknown >>>>>>> Source) >>>>>>> at >>>>>>> java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown >>>>>>> Source) >>>>>>> at >>>>>>> java.base/sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source) >>>>>>> at java.base/java.nio.file.Files.delete(Unknown Source) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.cacheLibrary(NativeLibLoader.java:300) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.installLibraryFromResource(NativeLibLoader.java:218) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.loadLibraryFromResource(NativeLibLoader.java:200) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.loadLibraryInternal(NativeLibLoader.java:142) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.loadLibrary(NativeLibLoader.java:58) >>>>>>> at >>>>>>> com.sun.prism.d3d.D3DPipeline.lambda$static$0(D3DPipeline.java:54) >>>>>>> at >>>>>>> java.base/java.security.AccessController.doPrivileged(Unknown Source) >>>>>>> at >>>>>>> com.sun.prism.d3d.D3DPipeline.<clinit>(D3DPipeline.java:50) >>>>>>> at java.base/java.lang.Class.forName0(Native Method) >>>>>>> at java.base/java.lang.Class.forName(Unknown Source) >>>>>>> at java.base/java.lang.Class.forName(Unknown Source) >>>>>>> at >>>>>>> com.sun.prism.GraphicsPipeline.createPipeline(GraphicsPipeline.java:218) >>>>>>> at >>>>>>> com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.init(QuantumRenderer.java:92) >>>>>>> at >>>>>>> com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:125) >>>>>>> at java.base/java.lang.Thread.run(Unknown Source) >>>>>>> GraphicsPipeline.createPipeline failed for >>>>>>> com.sun.prism.d3d.D3DPipeline >>>>>>> java.lang.UnsatisfiedLinkError: no prism_d3d in java.library.path: >>>>>>> C:\Program >>>>>>> Files\KafkIO;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program >>>>>>> Files\Oculus\Support\oculus-runtime;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;c:\dev\apps\apache-maven-3.9.9\bin;C:\dev\apps\cygwin64\bin;C:\Program >>>>>>> Files >>>>>>> (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program >>>>>>> Files\dotnet\;C:\Program Files\Git\cmd;C:\Program Files\7-Zip;C:\Program >>>>>>> Files\SafeNet\Authentication\SAC\x64;C:\Program >>>>>>> Files\SafeNet\Authentication\SAC\x32;C:\Program >>>>>>> Files\Docker\Docker\resources\bin;%JAVA_HOME%\bin;;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\ >>>>>>> xyz\scoop\apps\zulu-jdk\current\bin;C:\Users\xyz >>>>>>> \scoop\apps\zulu22-jdk\current\bin;C:\Users\xyz >>>>>>> \scoop\apps\zulu21-jdk\current\bin;C:\Users\xyz >>>>>>> \scoop\shims;C:\Users\xyz >>>>>>> \AppData\Local\Microsoft\WindowsApps;C:\dev\scripts;C:\Program >>>>>>> Files\JetBrains\IntelliJ IDEA 2024.2\bin;C:\Program Files\KafkIO\app;. >>>>>>> at java.base/java.lang.ClassLoader.loadLibrary(Unknown >>>>>>> Source) >>>>>>> at java.base/java.lang.Runtime.loadLibrary0(Unknown Source) >>>>>>> at java.base/java.lang.System.loadLibrary(Unknown Source) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.loadLibraryInternal(NativeLibLoader.java:170) >>>>>>> at >>>>>>> com.sun.glass.utils.NativeLibLoader.loadLibrary(NativeLibLoader.java:58) >>>>>>> at >>>>>>> com.sun.prism.d3d.D3DPipeline.lambda$static$0(D3DPipeline.java:54) >>>>>>> at >>>>>>> java.base/java.security.AccessController.doPrivileged(Unknown Source) >>>>>>> at >>>>>>> com.sun.prism.d3d.D3DPipeline.<clinit>(D3DPipeline.java:50) >>>>>>> at java.base/java.lang.Class.forName0(Native Method) >>>>>>> at java.base/java.lang.Class.forName(Unknown Source) >>>>>>> at java.base/java.lang.Class.forName(Unknown Source) >>>>>>> at >>>>>>> com.sun.prism.GraphicsPipeline.createPipeline(GraphicsPipeline.java:218) >>>>>>> at >>>>>>> com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.init(QuantumRenderer.java:92) >>>>>>> at >>>>>>> com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:125) >>>>>>> at java.base/java.lang.Thread.run(Unknown Source) >>>>>> >>>>>> >>>>>> >>>>>> Kind Regards, >>>>>> Cormac >>>>>> >>>>>> >>>>>> >>> >>> -- >>> Steve Hannah >>> Web Lite Solutions Corp. >>> >>
