On Sun, May 21, 2017 at 6:48 AM, Nicolai Hähnle <nhaeh...@gmail.com> wrote: > Hi all, > > I've been looking into ARB_gl_spirv for radeonsi. I don't fancy re-inventing > the ~8k LOC of src/compiler/spirv, and there's already a perfectly fine > SPIR-V -> NIR -> LLVM compiler pipeline in radv, so I looked into re-using > that. > > It's not entirely straightforward because radeonsi and radv use different > "ABIs" for their shaders, i.e. prolog/epilog shader parts, different user > SGPR allocations, descriptor loads work differently (obviously), and so on. > > Still, it's possible to separate the ABI from the meat of the NIR -> LLVM > translation. So here goes... > > > The Step-by-Step Plan > ===================== > > 1. Add an optional GLSL-to-NIR path (controlled by R600_DEBUG=nir) for very > simple VS-PS pipelines. > > 2. Add GL_ARB_gl_spirv support to Mesa and test it on simple VS-PS > pipelines. > > 3. Fill in all the rest: > 3a. GL 4.x shader extensions (SSBOs, images, atomics, ...) > 3b. Geometry and tessellation shaders > 3c. Compute shaders > 3d. Tests > > I've started with step 1 and got basic GLSL 1.30-level vertex shaders > working via NIR. The code is here: > https://cgit.freedesktop.org/~nh/mesa/log/?h=nir > > The basic approach is to introduce `struct ac_shader_abi' to capture the > differences between radeonsi and radv. In the end, the entry point for NIR > -> LLVM translation will simply be: > > void ac_nir_translate(struct ac_llvm_context *ac, > struct ac_shader_abi *abi, > struct nir_shader *nir); > > Setting up the LLVM function with its parameters is still considered part of > the driver. > > > Questions > ========= > > 1. How do we get good test coverage? > ------------------------------------ > A natural candidate would be to add a SPIR-V execution mode for the piglit > shader_runner. That is, use build scripts to extract shaders from > shader_test files and feed them through glslang to get spv files, and then > load those from shader_runner if a `-spirv' flag is passed on the command > line. > > This immediately runs into the difficulty that GL_ARB_gl_spirv wants SSO > linking semantics, and I'm pretty sure the majority of shader_test files > don't support that -- if only because they don't set a location on the > fragment shader color output. > > Some ideas: > 1. Add a GL_MESA_spirv_link_by_name extension > 2. Have glslang add the locations for us (probably difficult because glslang > seems to be focused on one shader stage at a time.) > 3. Hack something together in the shader_test-to-spv build scripts via > regular expressions (and now we have two problems? :-) ) > 4. Other ideas? > > > 2. What's the Gallium interface? > -------------------------------- > Specifically, does it pass SPIR-V or NIR? > > I'm leaning towards NIR, because then specialization, mapping of uniform > locations, atomics, etc. can be done entirely in st/mesa. > > On the other hand, Pierre Moreau's work passes SPIR-V directly. On the third > hand, it wouldn't be the first time that clover does things differently. > > > 3. NIR vs. TGSI > --------------- > It is *not* a goal for this project to use NIR for normal GLSL shaders. > We'll keep the TGSI backend at least for now. But it makes sense to think > ahead. > > A minor disadvantage of NIR is that the GLSL-to-NIR path is not as solid as > the GLSL-to-TGSI path yet, but this shouldn't be too difficult to overcome.
I *think* wiring up geom/tess support for glsl_to_nir should be straightforward, since like compute shaders, mesa/st is not creating any shader variants for these. I mostly just haven't done this yet since I don't have a way to test this yet. But it's not like we don't have some radeon hw around the office, so I expect I could help some here if needed. > The major disadvantage of NIR is that it doesn't have serialization. > radeonsi uses the fact that TGSI *is* a serialization format for two things: > > - The internal shader cache, which avoids re-compiling the same shader over > and over again when it's linked into different programs. (This part only > needs a strong hash.) > > - The (disk) shader cache stores the TGSI so that it's available in case > additional shader variants need to be compiled on the fly. > > Some ideas: > > 1. Add a serialization format for NIR. This is the most straight-forward > solution, but it's a lot of work for a comparatively small feature. > > 1b. Use SPIR-V as a serialization format for NIR. This is more work for > serialization than a custom format due to the ceremony involved in SPIR-V, > but we already have deserialization. Also, it'd implicitly give us an > alternative GLSL-to-SPIR-V compiler, which is kind of neat. > > 2. Don't store TGSI/NIR in the (disk) shader cache. The reason we have to do > that right now is that radeonsi does multi-threaded compilation, and so we > cannot fallback all the way back to GLSL compilation if we need to compile a > new shader variant. However, once we properly implement > ARB_parallel_shader_compile, this issue will go away. > > This doesn't address the internal shader cache, though. > > 3. Have st/mesa recognize when the same shader is linked into multiple > programs and avoid generating duplicate shader CSOs where possible. This is > non-trivial mostly because linking can map shader I/O into different places, > but I imagine that it would cover the majority of the cases caught by > radeonsi's internal shader cache. > > 4. Something else for the internal shader cache? > > 5. Use TGSI after all. TGSI really isn't such a bad format. It does have > some warts, like pretending that everything is a vec4, which means that f64 > support is already annoying, f16 support is going to be annoying, the way we > do UBOs should probably be re-written (since it cannot support std430 > packing, which would be nice to have), and real (non-inline) function > support will be nasty if we ever get there. But TGSI works, it's patient and > straightforward. > > That said, NIR is nicer in several ways. Not using it just because it can't > do serialization would be sad, not to mention those ~8k LOCs of > SPIRV-to-NIR. We could go SPIRV-to-NIR-to-TGSI, of course, but that's not > exactly great for compiler performance. I guess fleshing out the missing bits of nir_to_tgsi would be more work than the missing bits for mesa/st using glsl_to_nir.. and at least with pipe_shader_state now we can have both spirv->nir->driver and glsl->tgsi->driver in parallel. (and even if serializing the intermediate was needed, that might not be significantly more work than nir_to_tgsi.. and *probably* more useful to other drivers) btw, one vaguely related thing is spirv for compute.. afaiu for opencl compute shaders spirv is not required to have OpLoopMerge/OpSelectionMerge and structured CFG, which I think will make conversion into nir a bit more tricky. I guess doesn't matter for GL_ARB_gl_spirv, but might be nice in the long term to make clover work more similarly to other state trackers. I think I'll want that eventually for freedreno, although I think first I should finish up some things in nir->ir3 so I can finally switch nir on by default for VS/FS (since tgsi_to_nir does not support SSBOs/images/geom/tess/compute). BR, -R > All of this doesn't necessarily block the project of adding GL_ARB_gl_spirv, > but it'd be nice to think a bit ahead. > > So, what does everybody think? I'm particularly interested in the nouveau > folks' take on the whole NIR vs. TGSI thing, and any ideas on how to address > the above questions. > > Cheers, > Nicolai > -- > Lerne, wie die Welt wirklich ist, > Aber vergiss niemals, wie sie sein sollte. > _______________________________________________ > mesa-dev mailing list > mesa-dev@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/mesa-dev _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
