> I'm not sure what you mean by this. If you mean the current sandbox code that > uses JNI will need to be rewritten, then yes, that will be a rewrite, which > is why I advocate to take advantage of FFM sooner rather than later. If > you're mostly copying from D3D9 then it does save some effort on the JNI > side. D3D12 (or any DLL) doesn't know what's calling it, whether it's C++ or > Java, so there shouldn't be a difference in "integration". At least not when > I called native libraries, but perhaps I'm missing something. I can also give > a concrete example on some code snippet in our D3D9 or 12 if it helps clarify > my intention.
I understand that, I think I had to word that poorly earlier on. My point is that 12’s hardware abstraction is much thinner compared to 9 or even 11. This exposes some inner workings of GPUs which now have to be accommodated by the application rather than the GPU driver. Of course I don’t disagree that you can do this in Java, I suspect with proper bindings you could also write a working backend in any language you pick, no matter how high or low level. It is just my prior experience with 12, it makes a bit more sense to handle those fully natively from C/C++ code. > As for what to move to the Java side, there could be a performance tradeoff > between an increased number of native calls and the ability of the JIT to > optimize them. Regarding performance implications, it is true that this current approach reduces calls to native side, and I would even say to a quite significant margin in some cases. As an example, take one simple nRenderQuads() call - once we internally batch enough quads and Ring Buffer for them is used up enough, it submits current Command List onto the GPU for execution. However, because D3D12 is essentially context-less compared to older D3D versions or ES2, we need to start the next Command List by re-recording some initial GPU commands that will “restore” the pipeline before we can call another Draw. Otherwise bad things will happen, especially with Prism expecting a context-based approach in the first place. This is all done by D3D12RenderingContext.cpp and friends. The whole reason this is done is to parallelize GPU and CPU work as much as possible (although this still needs fine-tuning to truly maximize the FPS, but I digress). D3D9, D3D11, ES2 – in all these the driver does that for you, in 12 (and Metal) you have to do this on your own if your application needs to (and in our case it does). Similar situation with NativeBuffer I mentioned in my previous message - if we want to create a D3D12 Resource that’s allocated directly on VRAM and fill it with data, we need to do a whole lot of work, including a manual GPU call to copy the data to VRAM. Simplifying it to a simple Init() call that can be called from Java side just makes sense. > One of FFM's goals is to eliminate the need to create a bridge DLL like > prism_d3d12. I don't know if it's technically feasible with regard to how > native memory management works and because it works with C and not completely > with C++. Technically, D3D12 is purely a C++ API which also integrates a bit with Windows API for classic HWND handles and for synchronization with Windows’ Event Handles. I do still access it despite JNI requiring C-style entry points, but there are some technicalities to handle with how and when you can free 12’s resources (mostly to make sure you don’t free them while GPU is still using them). Again, I don’t deny that we can do all this work I mentioned above in Java. My point is, skipping the bridge DLL, while possible and might have some Java advantages, still seems like not the most feasible approach with what D3D12 demands from the application. And there’s still the licensing side (forgot that word existed last message… the wonders of being ESL =) ) of jextract-ing D3D12 headers, which I am definitely not an expert on. If we do decide to use FFM, I would rather expect using it with current structure – to access built-by-us prism_d3d12.dll and to call APIs that it exposes. But, all that is still not a priority for this backend right now - we might take a look at it later down the line. Lukasz From: Nir Lisker <nlis...@gmail.com> Sent: Thursday, 17 October 2024 18:33 To: Lukasz Kostyra <lukasz.kost...@oracle.com> Cc: openjfx-dev <openjfx-dev@openjdk.org> Subject: Re: [External] : Re: JavaFX Direct3D 12 rendering pipeline for Windows Hi Lukasz, Yes, FFM is quite a bit to take in. I haven't mastered the newest version myself yet. In many cases I mostly followed D3D9 backend's design, so the similarities can be there. =) But there definitely is room for improvement which I also see and would like to tackle before the backend reaches a "mature enough" state. When working on 3D part of JFX I also saw that there is some data duplication happening, especially with Light/Mesh/MeshView code. I'd like to trim this as well, but it was not a priority yet - I preferred to have a working code now and all sorts of cleanups or reorganizations like that can come later. Regarding FFM - that is an interesting idea that we could explore at some point, but I'm not sure if we could (or even should) move all the implementation details to Java and limit native calls to just D3D12. There are a few problems with that: Because FFM saves a lot of handwriting of fragile JNI code, the sooner it's adopted, the less headache you (or collaborators) will have. So while it's true that there's a minimal working level the code should be brought to, adopting FFM will mean rewriting a lot of JNI code. This is why I suggest going with FFM as soon as that minimal level is reached. It has advantages in type safety and compiler checks that JNI does not have, including the ability to use objects-structs mapping instead of breaking them down to JNI-supported primitives (long -> jlong etc.). - There is quite a bit of logic which integrates better on C/C++ side with D3D12, mostly due to 12's low-level-ness. Rewriting those in Java feels like quite a bit of a task in itself, and might create more problems than it solves. I'm not sure what you mean by this. If you mean the current sandbox code that uses JNI will need to be rewritten, then yes, that will be a rewrite, which is why I advocate to take advantage of FFM sooner rather than later. If you're mostly copying from D3D9 then it does save some effort on the JNI side. D3D12 (or any DLL) doesn't know what's calling it, whether it's C++ or Java, so there shouldn't be a difference in "integration". At least not when I called native libraries, but perhaps I'm missing something. I can also give a concrete example on some code snippet in our D3D9 or 12 if it helps clarify my intention. - Raising the abstraction from 12’s low-level approach instead of just making a “passthrough” is also something that’s quite frequently done, mostly to make the API a bit more useable and suitable for project’s specific needs (see ex. NativeBuffer abstractifying 12’s heap types into simply data being CPU visible, or code managing mid-frame data like CommandLists, shader constants, 2D vertices etc.). This could also help later down the line when adding potential new features, simplifying the calls made from Java side. I'm not sure what this means either :p The abstraction that exists or will exist in C/C++ that surrounds the D3D API can exist instead in Java. It's the same abstraction level. - There might be some implications related to using jextract on third-party headers that we are unsure of and would require additional approvals before proceeding with this specific approach. OK, I don't know how this works with licensing if that's what you mean. I've used jextract (and even wrote a JavaFX GUI wrapper for it) on 3rd party headers and technically didn't see many issues. The most reasonable/probable FFM use would be to access a D3D12 backend DLL we create (so basically accessing current prism_d3d12.dll but with FFM and not JNI). But it is something that we can explore in the future, for now I think there are more higher-priority tasks related to this backend. One of FFM's goals is to eliminate the need to create a bridge DLL like prism_d3d12. I don't know if it's technically feasible with regard to how native memory management works and because it works with C and not completely with C++. As for what to move to the Java side, there could be a performance tradeoff between an increased number of native calls and the ability of the JIT to optimize them. There are tricks there that we could use. There will be quite a bit to explore when the minimal working state is reached. By the way, Metal could use it too if it's not very far out, but I certainly would not replace the current ES and D3D9 code with FFM. On Thu, Oct 17, 2024 at 5:04 PM Lukasz Kostyra <lukasz.kost...@oracle.com<mailto:lukasz.kost...@oracle.com>> wrote: Hi Nir, Thanks for suggestions! Apologies for being a bit late, needed to wrap my head around FFM before responding. In many cases I mostly followed D3D9 backend's design, so the similarities can be there. =) But there definitely is room for improvement which I also see and would like to tackle before the backend reaches a "mature enough" state. When working on 3D part of JFX I also saw that there is some data duplication happening, especially with Light/Mesh/MeshView code. I'd like to trim this as well, but it was not a priority yet - I preferred to have a working code now and all sorts of cleanups or reorganizations like that can come later. Regarding FFM - that is an interesting idea that we could explore at some point, but I'm not sure if we could (or even should) move all the implementation details to Java and limit native calls to just D3D12. There are a few problems with that: - There is quite a bit of logic which integrates better on C/C++ side with D3D12, mostly due to 12's low-level-ness. Rewriting those in Java feels like quite a bit of a task in itself, and might create more problems than it solves. - Raising the abstraction from 12’s low-level approach instead of just making a “passthrough” is also something that’s quite frequently done, mostly to make the API a bit more useable and suitable for project’s specific needs (see ex. NativeBuffer abstractifying 12’s heap types into simply data being CPU visible, or code managing mid-frame data like CommandLists, shader constants, 2D vertices etc.). This could also help later down the line when adding potential new features, simplifying the calls made from Java side. - There might be some implications related to using jextract on third-party headers that we are unsure of and would require additional approvals before proceeding with this specific approach. The most reasonable/probable FFM use would be to access a D3D12 backend DLL we create (so basically accessing current prism_d3d12.dll but with FFM and not JNI). But it is something that we can explore in the future, for now I think there are more higher-priority tasks related to this backend. Lukasz From: Nir Lisker <nlis...@gmail.com<mailto:nlis...@gmail.com>> Sent: Tuesday, 15 October 2024 14:28 To: Lukasz Kostyra <lukasz.kost...@oracle.com<mailto:lukasz.kost...@oracle.com>> Cc: openjfx-dev <openjfx-dev@openjdk.org<mailto:openjfx-dev@openjdk.org>> Subject: [External] : Re: JavaFX Direct3D 12 rendering pipeline for Windows This is exciting news! I looked at the code quickly. Here are my thoughts. My biggest remark is that JNI is still being used, which is being more and more restricted. Recently, Kevin announced that we will be FFM-ready, surely by the time this pipeline is released. I would like to see (and can help, having used FFM since incubation) D3D12 being based on FFM. This will also allow less cluttered code to be written with less effort. Consider this mess in D3D12NativeMeshView (exists in D3D9 also, not blaming you for this :) ): private native void nSetLight(long ptr, int index, float x, float y, float z, float r, float g, float b, float enabled, float ca, float la, float qa, float isAttenuated, float maxRange, float dirX, float dirY, float dirZ, float innerAngle, float outerAngle, float falloff); That, in a pure Java world, would be `void nSetLight(Light)` with `Light` holding all its parameters. In fact, this could make whole files that just prepare the data for the native layer redundant. At least I know this is the case in the current D3D9 implementation. It might also be the case that the pointer juggling that we do currently (using `long ptr` everywhere) will not be needed because FFM does it for us (Arena takes care of the lifecycle context). However, when it comes to resource deallocation there are many caveats, so I'm speculating here. Another design element that I think we can get rid of is the duplication of data structures in the Java and native sides. if Java has a Light, Material, and Mesh, the native side doesn't need them in reality, but in D3D9 (and somewhat in D3D12) it does have them. Without checking, I suspect that this duplication leads to inefficient memory usage. I speculate that it was done to minimize the number of native calls, a consideration that could be overturned with FFM. In general, a lot of the code in C++ can be potentially moved to Java, with only the calls to DirectX being left of course. Thanks for this herculean effort! On Mon, Oct 14, 2024 at 7:10 PM Lukasz Kostyra <lukasz.kost...@oracle.com<mailto:lukasz.kost...@oracle.com>> wrote: Hello openjfx-dev, we just pushed a prototype of a new JavaFX Direct3D 12 rendering pipeline for Windows to a new "direct3d12" branch on jfx-sandbox. It is more than an experiment branch - we intend to fully develop the D3D12 backend there. We're not necessarily looking for contributions at this point, but if anyone has early feedback about it or wants to try it by building it themselves, that would be fine. We also did not test it on a wider range of hardware, so your mileage may vary. While D3D12 pipeline will build by default, D3D9 pipeline is still the default pick at runtime. To run anything on D3D12 pipeline you need to force it with ex.: java -Dprism.order=d3d12 ... Backend supports 2D rendering (albeit with some graphical issues here and there that need to be ironed out) and basic 3D rendering. Expect not everything fully working yet (ex. some gradients on 2D controls are incorrect, or 3D-in-2D will straight up not work) and the performance not matching D3D9 yet. Our goal is to first reach feature completion and then focus on performance. Lukasz
