[Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
I’m writing a custom NVMe device model and I need to test ATS functionality. For that, I need to enable the IOMMU. I start QEMU with the following command line, using a stock Ubuntu 16.04 image: % emu-system-x86_64 -M q35,accel=kvm,kernel_irqchip=off -m 8K -device intel-iommu,intremap=on,device-iotlb=on -device myNVMe,addr=10.0 It boots just fine and I see the /dev/nvme* device files, but the addresses provided for DMA transfers are the same as without a IOMMU. I see there are no iommu groups defined, and thus my device is obviously not assigned to any one of them. DMAR-related syslog messages: ACPI: DMAR 0x7FFE20C5 50 (v01 BOCHS. BXPCDMAR 0001 BXPC 0001) DMAR: Host address width 39 DMAR: DRHD base: 0x00fed9 flags: 0x1 DMAR: dmar0: reg_base_addr fed9 ver 1.0 cap 12008c22260206 cap f00f4e DMAR: ATSR flash: 0x1 DMAR-IR: IOAPIC id 0 under DRHD base. 0xfed9 IOMMU 0 DMAR-IR: Enabled IRQ remapping in Salic mode How can I get the DMA buffers use virtual addresses?
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
Sorry, transcription error. Yes, I used intel-iommu=on on the guest command line.
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
Correction to my correction: no “=on”. If I add “=on, I get ‘Parameter ‘driver’ missing' > On Aug 21, 2019, at 14:15, Paul Lancaster wrote: > > Sorry, transcription error. Yes, I used intel-iommu=on on the guest command > line. >
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
On Wed, 2019-08-21 at 13:37 -0700, Paul Lancaster wrote: > I’m writing a custom NVMe device model and I need to test ATS functionality. > For that, I need to enable the IOMMU. > > I start QEMU with the following command line, using a stock Ubuntu 16.04 > image: > > % emu-system-x86_64 -M q35,accel=kvm,kernel_irqchip=off -m 8K -device > intel-iommu,intremap=on,device-iotlb=on -device myNVMe,addr=10.0 > > It boots just fine and I see the /dev/nvme* device files, but the addresses > provided for DMA transfers are the same as without a IOMMU. > > I see there are no iommu groups defined, and thus my device is obviously not > assigned to any one of them. > > DMAR-related syslog messages: > > ACPI: DMAR 0x7FFE20C5 50 (v01 BOCHS. BXPCDMAR 0001 BXPC > 0001) > DMAR: Host address width 39 > DMAR: DRHD base: 0x00fed9 flags: 0x1 > DMAR: dmar0: reg_base_addr fed9 ver 1.0 cap 12008c22260206 cap f00f4e > DMAR: ATSR flash: 0x1 > DMAR-IR: IOAPIC id 0 under DRHD base. 0xfed9 IOMMU 0 > DMAR-IR: Enabled IRQ remapping in Salic mode > > How can I get the DMA buffers use virtual addresses? > > Did you enable the iommu in the guest? You need intel_iommu=on on guest't command line to enable the virtual IOMMU. And you might need iommu=on (usually this is the default, can be changed by Kconfig option) (it can be 'on' or 'pt', and later makes kernel driver use passthrough mode in which too as you say the addresses provided for DMA transfers would be the same as without a IOMMU) Best regards, Maxim Levitsky
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
Doh! You meant iommu_intel=on kernel argument to the guest boot command line… (sorry, I’m a HW guy, not a linux/kernel guy). Just did. No change.
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
On Wed, 2019-08-21 at 14:17 -0700, Paul Lancaster wrote: > Correction to my correction: no “=on”. If I add “=on, I get ‘Parameter > ‘driver’ missing' > > > On Aug 21, 2019, at 14:15, Paul Lancaster > > wrote: > > > > Sorry, transcription error. Yes, I used intel-iommu=on on the guest command > > line. > > > > No no! not on qemu command line, but on the guest's kernel command line. You need to boot the guest, and update the kernel command line that grub uses in the guest. Best regards, Maxim Levitsky
Re: [Qemu-discuss] IOMMU and custom PCIe Device in non-nested VMs
On Wed, 2019-08-21 at 14:30 -0700, Paul Lancaster wrote: > Doh! You meant iommu_intel=on kernel argument to the guest boot command line… > (sorry, I’m a HW guy, not a linux/kernel guy). > > Just did. No change. You need intel_iommu=on on the guest's command line. Best regards, Maxim Levitsky
