Hi all, I'm trying to pass a LSI SAS2008 controller to a guest using VFIO. First of all, some details on my system:
# lsb_release -d
Description: Ubuntu 14.04.4 LTS
# uname -rs
Linux 4.4.0-22-generic
# cat /proc/cpuinfo | grep -m 1 "model name"
model name : Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz
# qemu-system-x86_64 --version
QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright (c)
2003-2008 Fabrice Bellard
# libvirtd --version
libvirtd (libvirt) 1.2.2
I'm using libvirt/virsh to pass the device to a (freenas) guest and at first it
looks like the domain would start just fine because virsh reports "Domain
freeness started". The domain, however, crashes right after start with the
following message from `/var/log/libvirt/qemu/freenas.log`. The full log is
attached:
qemu-system-x86_64: vfio_dma_map(0x55b0a94b63c0, 0xfeb90000, 0x2000,
0x7f22ca700000) = -14 (Bad address)
qemu: hardware error: vfio: DMA mapping failed, unable to continue
The LSI controller is in IOMMU group 1 and has the id `01:00.0`. It shares the
group with a PCI bridge:
------------------
| IOMMU group 1 |
------------------
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core
Processor PCI Express x16 Controller [8086:0c01] (rev 06)
Kernel driver in use: pcieport
01:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic
SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
Kernel driver in use: vfio-pci
The full IOMMU group distribution is attached, too.
I'm currently assigning the device manually to vfio-pci. The relevant part of
the domain's XML configuration is:
<hostdev mode='subsystem' type='pci' managed='no'>
<driver name='vfio'/>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
</hostdev>
The problem only seems to occur with the LSI controller `01:00.0`. Passing
another device in a non-singleton IOMMU group, e. g. the Firewire device
`05:03.0` in group 17, works fine (even with `managed='yes'`):
------------------
| IOMMU group 17 |
------------------
04:00.0 PCI bridge [0604]: Tundra Semiconductor Corp. Device [10e3:8113]
(rev 01)
05:03.0 FireWire (IEEE 1394) [0c00]: Texas Instruments TSB43AB22A
IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx] [104c:8023]
Kernel driver in use: firewire_ohci
Previously, I used to pass the controller using KVM pci assignment but due to
it's deprecation in Linux 4.2 I wanted to switch to VFIO. Generally, it seems
to work on my system but not for the device in question.
I'd appreciate any help.
Best,
Damon
libvirt-qemu-freenas.log
Description: Binary data
------------------
| IOMMU group 0 |
------------------
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v3 Processor DRAM
Controller [8086:0c08] (rev 06)
Kernel driver in use: ie31200_edac
------------------
| IOMMU group 1 |
------------------
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core
Processor PCI Express x16 Controller [8086:0c01] (rev 06)
Kernel driver in use: pcieport
01:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic
SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
Kernel driver in use: vfio-pci
------------------
| IOMMU group 2 |
------------------
00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3
Processor Integrated Graphics Controller [8086:041a] (rev 06)
Kernel driver in use: i915
------------------
| IOMMU group 3 |
------------------
00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core
Processor HD Audio Controller [8086:0c0c] (rev 06)
Kernel driver in use: snd_hda_intel
------------------
| IOMMU group 4 |
------------------
00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset
Family USB xHCI [8086:8c31] (rev 05)
Kernel driver in use: xhci_hcd
------------------
| IOMMU group 5 |
------------------
00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series
Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
Kernel driver in use: mei_me
00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series
Chipset Family KT Controller [8086:8c3d] (rev 04)
Kernel driver in use: serial
------------------
| IOMMU group 6 |
------------------
00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection
I217-LM [8086:153a] (rev 05)
Kernel driver in use: e1000e
------------------
| IOMMU group 7 |
------------------
00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset
Family USB EHCI #2 [8086:8c2d] (rev 05)
Kernel driver in use: ehci-pci
------------------
| IOMMU group 8 |
------------------
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset
High Definition Audio Controller [8086:8c20] (rev 05)
Kernel driver in use: snd_hda_intel
------------------
| IOMMU group 9 |
------------------
00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #1 [8086:8c10] (rev d5)
Kernel driver in use: pcieport
------------------
| IOMMU group 10 |
------------------
00:1c.3 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #4 [8086:8c16] (rev d5)
Kernel driver in use: pcieport
------------------
| IOMMU group 11 |
------------------
00:1c.5 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #6 [8086:8c1a] (rev d5)
Kernel driver in use: pcieport
------------------
| IOMMU group 12 |
------------------
00:1c.6 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #7 [8086:8c1c] (rev d5)
Kernel driver in use: pcieport
------------------
| IOMMU group 13 |
------------------
00:1c.7 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #8 [8086:8c1e] (rev d5)
Kernel driver in use: pcieport
------------------
| IOMMU group 14 |
------------------
00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset
Family USB EHCI #1 [8086:8c26] (rev 05)
Kernel driver in use: ehci-pci
------------------
| IOMMU group 15 |
------------------
00:1f.0 ISA bridge [0601]: Intel Corporation C226 Series Chipset Family Server
Advanced SKU LPC Controller [8086:8c56] (rev 05)
Kernel driver in use: lpc_ich
00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset
Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05)
Kernel driver in use: ahci
00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family
SMBus Controller [8086:8c22] (rev 05)
00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset
Family Thermal Management Controller [8086:8c24] (rev 05)
------------------
| IOMMU group 16 |
------------------
03:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network
Connection [8086:1533] (rev 03)
Kernel driver in use: igb
------------------
| IOMMU group 17 |
------------------
04:00.0 PCI bridge [0604]: Tundra Semiconductor Corp. Device [10e3:8113] (rev
01)
05:03.0 FireWire (IEEE 1394) [0c00]: Texas Instruments TSB43AB22A
IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx] [104c:8023]
Kernel driver in use: firewire_ohci
------------------
| IOMMU group 18 |
------------------
06:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host
Controller [1912:0015] (rev 02)
Kernel driver in use: xhci_hcd
------------------
| IOMMU group 19 |
------------------
07:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA
Controller [1b21:0612] (rev 01)
Kernel driver in use: ahci
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
