I read into this a bit more...for amd gpus apparently there is some fpsimd thing needed for at least DCN cards[1]. Also dml is compiled with -mgeneral-regs-only on aarch64 on the linux driver.
[1] https://lists.freedesktop.org/archives/amd-gfx/2022-October/086009.html
