Begin forwarded message:
From: Xu Liu <li...@meta.com>
Subject: How do I make my emulated device's DMA go through viommu ?
Date: February 14, 2024 at 11:46:04 AM EST
To: "qemu-disc...@nongnu.org" <qemu-disc...@nongnu.org>
Cc: 刘旭 <liuxu1...@gmail.com>, Xu Liu <li...@meta.com>
Hello, All
I am working a project, which has a custom emulated PCI device for the QEMU.
In side the emulated device, I call pci_dma_read/write to write to some IOVA
address, for example 0X400000030000.
After start the QEMU, I bind the emulated device to vfio-pci driver .
It always complains:
Invalid access at addr 0x400000030000, size 4, region '(null)', reason: rejected
From my application log, I can see the IOVA is mapped:
{ iova_range: RangeInclusive { start: 70368744374272, end: 70368744378367 },
memfd: Memfd { file: File { fd: 167, path: "/memfd:dma_memory_0x400000030000
(deleted)", read: true, write: true } }, mmap: MmapRaw { ptr: 0x7feb9bc7a000,
len: 4096 } }
From the QEMU OS kernel trace, I can see the mapping is there:
workload-697 [000] ..... 31.476109: map: IOMMU: iova=0x0000400000030000 -
0x0000400000031000 paddr=0x0000000109eca000 size=4096
From QEMU trace event, I can see the IOMMU is enabled. the 02:00.0 is a PCI
bridge, and 03:00.0 is my emulated device
pci_update_mappings_add d=0x56461fcf1a40 03:00.0 0,0xfe800000+0x100000
pci_update_mappings_add d=0x56461fcf1a40 03:00.0 1,0xfe900000+0x100000
pci_update_mappings_add d=0x56461fcf1a40 03:00.0 2,0x8000000000+0x1000000
pci_update_mappings_add d=0x56461fcf1a40 03:00.0 4,0x4000000000+0x4000000000
vtd_switch_address_space Device 02:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 02:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 03:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 03:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 02:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 02:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 03:00.0 switching address space (iommu
enabled=1)
vtd_switch_address_space Device 03:00.0 switching address space (iommu
enabled=1)
From GDB, looks like the IOVA is io_mem_unassigned:
$20 = {size = 18446744073709551616, mr = 0x5613a6519840 <io_mem_unassigned>, fv
= 0x5613a67ab3e0, offset_within_region = 0, offset_within_address_space = 0,
readonly = false, nonvolatile = false}
Any ideas about this ?
Thanks.
Xu
