On Wed, Aug 06, 2025 at 11:11:23PM +0800, Tao Tang wrote: > Hi all, > > This patch series introduces initial support for emulating the Arm SMMUv3 > Secure State. > > As Pierrick pointed out in a previous discussion [1], full Secure SMMUv3 > emulation is a notable missing piece in QEMU. While the FVP model has > some support, its limited PCIe capabilities make it challenging for > complex use cases. The ability to properly manage device DMA from a > secure context is a critical prerequisite for enabling device assignment > (passthrough) for confidential computing solutions like Arm CCA and > related research such as virtCCA [2]. This series aims to build that > foundational support in QEMU. > > This work is being proposed as an RFC. It introduces a significant amount > of new logic, including the core concept of modeling parallel secure and > non-secure contexts within a single SMMUv3 device. I am seeking feedback > on the overall approach, the core refactoring, and the implementation > details before proceeding further. > > The series begins by implementing the components of the secure programming > interface, then progressively refactors the core SMMU logic to handle > secure and non-secure contexts in parallel. > > Secure Interface Implementation: The initial patches add the > secure-side registers, implement their read/write logic, and enable > the secure command and event queues. This includes the S_INIT > mechanism and the new secure TLB invalidation commands. > > Core Logic Refactoring: The next set of patches makes the core SMMU > functions security-state aware. This involves plumbing an is_secure > context flag through the main code paths and adding logic to route > SMMU-originated memory accesses to the correct (Secure or Non-secure) > address space. > > Cache Isolation: With the core logic now aware of security states, > the following patches refactor the configuration and translation > lookup caches. The cache keys are modified to include the security > context, ensuring that secure and non-secure entries for the same > device or address are properly isolated and preventing aliasing. > > Framework Integration: The final patch connects the SMMU's internal > security context to the generic QEMU IOMMU framework by using the > iommu_index to represent the architectural SEC_SID. > > To validate this work, I performed the following tests: > > Non-Secure Regression: To ensure that existing functionality remains > intact, I ran a nested virtualization test. A TCG guest was created on > the host, with iommu=smmuv3 and with an emulated PCIe NVMe device assigned. > Command line of TCG VM is below: > > qemu-system-aarch64 \ > -machine virt,virtualization=on,gic-version=3,iommu=smmuv3 \ > -cpu max -smp 1 -m 4080M \ > -accel tcg,thread=single,tb-size=512 \ > -kernel Image \ > -append 'nokaslr root=/dev/vda rw rootfstype=ext4 iommu.passthrough=on' \ > -device pcie-root-port,bus=pcie.0,id=rp0,addr=0x4.0,chassis=1,port=0x10 \ > -device pcie-root-port,bus=pcie.0,id=rp1,addr=0x5.0,chassis=2,port=0x11 \ > -drive if=none,file=u2204fs.img.qcow2,format=qcow2,id=hd0 \ > -device virtio-blk-device,drive=hd0 \ > -qmp unix:/tmp/qmp-sock12,server=on,wait=off \ > -netdev user,id=eth0,hostfwd=tcp::10022-:22,hostfwd=tcp::59922-:5922 \ > -device virtio-net-device,netdev=eth0 \ > -drive if=none,file=nvme.img,format=raw,id=nvme0 \ > -device nvme,drive=nvme0,serial=deadbeef \ > -d unimp,guest_errors -trace events=smmu-events.txt -D qemu.log -nographic > > Inside this TCG VM, a KVM guest was launched, and the same NVMe device was > re-assigned to it via VFIO. > Command line of KVM VM inside TCG VM is below: > > sudo qemu-system-aarch64 \ > -enable-kvm -m 1024 -cpu host -M virt \ > -machine virt,gic-version=3 \ > -cpu max -append "nokaslr" -smp 1 \ > -monitor stdio \ > -kernel 5.15.Image \ > -initrd rootfs.cpio.gz \ > -display vnc=:22,id=primary \ > -device vfio-pci,host=00:01.0 > > The KVM guest was able to perform I/O on the device > correctly, confirming that the non-secure path is not broken.
I gave the patches a quick test and they seem to have broken my nested setup, I will look more into it and let you know what I find. Thanks, Mostafa > > Secure Register/Command Interface: I set up an OP-TEE + Hafnium > environment. Hafnium's smmuv3_driver_init function was used to test > the secure register I/O and command queue functionality (excluding > translation). As Hafnium assumes larger queue and StreamID sizes than > are practical without TTST support, I temporarily patched Hafnium to > use smaller values, allowing its driver to initialize the emulated > secure SMMU successfully. > > Secure Translation Path: Since the TF-A SMMUv3 Test Engine does not > support QEMU, and no secure device assignment feature exists yet, I > created a custom platform device to test the secure translation flow. > To trigger the translation logic, I initiated MMIO writes to this > device from within Hafnium. The device's MMIO callback handler then > performed DMA accesses via its IOMMU region, exercising the secure > translation path. While SMMUv3 is typically used for PCIe on > physical SoCs, the architecture allows its use with platform devices > via a stream-id binding in the device tree. The test harness > required some non-standard modifications to decouple the SMMU from > its tight integration with PCIe. The code for this test device is > available for review at [3]. README.md with detailed instructions is > also provided. > > I've attempted to follow all of the guidance in the "Submitting a Patch" > guide, but as this is my first series of this scale, I apologize if I > missed anything and welcome all feedback. > > Thanks, > Tang > > [1] https://lists.nongnu.org/archive/html/qemu-devel/2025-06/msg02940.html > [2] https://arxiv.org/abs/2306.11011 > [3] https://github.com/hnusdr/qemu > > Tao Tang (11): > hw/arm/smmuv3: Introduce secure registers and commands > hw/arm/smmuv3: Implement read/write logic for secure registers > hw/arm/smmuv3: Implement S_INIT for secure initialization > hw/arm/smmuv3: Enable command processing for the Secure state > hw/arm/smmuv3: Support secure event queue and error handling > hw/arm/smmuv3: Plumb security state through core functions > hw/arm/smmuv3: Add separate address space for secure SMMU accesses > hw/arm/smmuv3: Enable secure-side stage 2 TLB invalidations > hw/arm/smmuv3: Make the configuration cache security-state aware > hw/arm/smmuv3: Differentiate secure TLB entries via keying > hw/arm/smmuv3: Use iommu_index to represent SEC_SID > > hw/arm/smmu-common.c | 74 ++- > hw/arm/smmuv3-internal.h | 128 +++++- > hw/arm/smmuv3.c | 844 ++++++++++++++++++++++++++++++----- > hw/arm/trace-events | 7 +- > hw/arm/virt.c | 5 + > include/hw/arm/smmu-common.h | 23 +- > include/hw/arm/smmuv3.h | 27 ++ > 7 files changed, 968 insertions(+), 140 deletions(-) > > -- > 2.34.1 > >
