Adding that I got the device to pass, but there are errors. To start with, I found a post (including Alex Williamson's help) about finding the NHI device and binding that vfio. This approach worked. Currently my script to detach the devices and bind them to vfio is:
#!/bin/bash #NHI device virsh nodedev-detach pci_0000_00_0d_2 #OWC devices #SATA controllers virsh nodedev-detach pci_0000_03_00_0 virsh nodedev-detach pci_0000_04_00_0 #PCI Tbolt devices virsh nodedev-detach pci_0000_01_00_0 virsh nodedev-detach pci_0000_02_01_0 virsh nodedev-detach pci_0000_02_02_0 Followed by qemu: sudo qemu-system-x86_64 \ -vnc :0 \ -m 2048 -smp cores=2 \ -machine accel=kvm \ -drive file=/usr/share/OVMF/x64/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on \ -drive file=/home/chuwi-admin/OVMF_VARS.fd,if=pflash,format=raw,unit=1 \ -drive format=raw,file=/dev/sdb \ -device vfio-pci,host=03:00.0 \ -device vfio-pci,host=04:00.0 Now inside the VM running an Arch USB install image, the drives are detected. I am having trouble getting access to them. The initial dd /dev/zero to one of the disk devices errored out with timeouts and finally said it is a read only device. failed command: READ FPDMA QUEUED also revalidation failed Anytime I try to mount a disk, make a new fs on a disk, or basically do anything there are I/O errors. On Thu, Jun 10, 2021 at 5:59 PM kram threetwoone <kram...@gmail.com> wrote: > Hello everyone, and thanks for everything you do! > > The device is a OWC thunderbay 4. > https://www.owcdigital.com/products/thunderbay-4 > Inside it contains two SATA controllers ASM1062, and one Tbolt 3 Intel > DSL6540. > > The host is a Chuwi Corebox pro. > https://store.chuwi.com/products/corebox-pro > > I'll say I'm pretty familiar with GPU passthrough and have been doing it > since Oct 2014. The idea is to passthrough this OWC box to a headless > server via qemu/libvirt (no virt-manager). So far I have tried many > techniques such as: > > virsh nodedev-detach > manual modprobe vfio-pci > booting kernel with module vfio-pci ids=8086:15d3 (thunderbolt) and 1b21:1060 > (SATA controller) > > No matter what I do, the only success I have reached is having the Intel > controller show up without a driver in use on lspci -nnk, but the vfio > device /dev/vfio/XXX never get created. > > Please help, I'm new to passing through non-GPU devices. Thanks in > advance. For what it's worth, this exact box I have been able to > passthrough the Intel integrated GPU, so vfio is working. > > Any further information can be provided as requested, this is already too > much text. > > All following logs are generated as a stock system, everything reset to a > vanilla machine (no blacklists or vfio module loaded at boot): > LSPCI -nnk: > 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:8a02] (rev 03) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: icl_uncore > 00:02.0 VGA compatible controller [0300]: Intel Corporation Iris Plus > Graphics G1 (Ice Lake) [8086:8a56] (rev 07) > Subsystem: Device [1e50:800a] > Kernel driver in use: i915 > Kernel modules: i915 > 00:07.0 PCI bridge [0604]: Intel Corporation Ice Lake Thunderbolt 3 PCI > Express Root Port #0 [8086:8a1d] (rev 03) > Kernel driver in use: pcieport > 00:0d.0 USB controller [0c03]: Intel Corporation Ice Lake Thunderbolt 3 > USB Controller [8086:8a13] (rev 03) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: xhci_hcd > Kernel modules: xhci_pci > 00:0d.2 System peripheral [0880]: Intel Corporation Ice Lake Thunderbolt 3 > NHI #0 [8086:8a17] (rev 03) > Kernel driver in use: thunderbolt > Kernel modules: thunderbolt > 00:14.0 USB controller [0c03]: Intel Corporation Ice Lake-LP USB 3.1 xHCI > Host Controller [8086:34ed] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: xhci_hcd > Kernel modules: xhci_pci > 00:14.2 RAM memory [0500]: Intel Corporation Device [8086:34ef] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > 00:14.3 Network controller [0280]: Intel Corporation Killer Wi-Fi 6 > AX1650i 160MHz Wireless Network Adapter (201NGW) [8086:34f0] (rev 30) > Subsystem: Intel Corporation Device [8086:0074] > Kernel driver in use: iwlwifi > Kernel modules: iwlwifi > 00:15.0 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP Serial > IO I2C Controller #0 [8086:34e8] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: intel-lpss > Kernel modules: intel_lpss_pci > 00:16.0 Communication controller [0780]: Intel Corporation Management > Engine Interface [8086:34e0] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: mei_me > Kernel modules: mei_me > 00:17.0 SATA controller [0106]: Intel Corporation Ice Lake-LP SATA > Controller [AHCI mode] [8086:34d3] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: ahci > 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:34be] (rev 30) > Kernel driver in use: pcieport > 00:1c.7 PCI bridge [0604]: Intel Corporation Device [8086:34bf] (rev 30) > Kernel driver in use: pcieport > 00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:34b4] (rev 30) > Kernel driver in use: pcieport > 00:1e.0 Communication controller [0780]: Intel Corporation Ice Lake-LP > Serial IO UART Controller #0 [8086:34a8] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: intel-lpss > Kernel modules: intel_lpss_pci > 00:1e.3 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP Serial > IO SPI Controller #1 [8086:34ab] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: intel-lpss > Kernel modules: intel_lpss_pci > 00:1f.0 ISA bridge [0601]: Intel Corporation Ice Lake-LP LPC Controller > [8086:3482] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > 00:1f.3 Audio device [0403]: Intel Corporation Smart Sound Technology > Audio Controller [8086:34c8] (rev 30) > Subsystem: Realtek Semiconductor Co., Ltd. Device [10ec:119e] > Kernel driver in use: snd_hda_intel > Kernel modules: snd_hda_intel, snd_sof_pci_intel_icl > 00:1f.4 SMBus [0c05]: Intel Corporation Ice Lake-LP SMBus Controller > [8086:34a3] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > Kernel driver in use: i801_smbus > Kernel modules: i2c_i801 > 00:1f.5 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP SPI > Controller [8086:34a4] (rev 30) > Subsystem: Intel Corporation Device [8086:7270] > 01:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge > (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > Kernel driver in use: pcieport > 02:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge > (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > Kernel driver in use: pcieport > 02:02.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge > (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > Kernel driver in use: pcieport > 02:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge > (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > Kernel driver in use: pcieport > 03:00.0 SATA controller [0106]: ASMedia Technology Inc. Device [1b21:0625] > (rev 01) > Subsystem: ASMedia Technology Inc. Device [1b21:1060] > Kernel driver in use: ahci > 04:00.0 SATA controller [0106]: ASMedia Technology Inc. Device [1b21:0625] > (rev 01) > Subsystem: ASMedia Technology Inc. Device [1b21:1060] > Kernel driver in use: ahci > 2c:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev > 15) > Subsystem: Realtek Semiconductor Co., Ltd. Device [10ec:0123] > Kernel driver in use: r8169 > Kernel modules: r8169 > 2d:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev > 15) > Subsystem: Realtek Semiconductor Co., Ltd. Device [10ec:0123] > Kernel driver in use: r8169 > Kernel modules: r8169 > 2e:00.0 Non-Volatile memory controller [0108]: Kingston Technology > Company, Inc. Device [2646:5008] (rev 01) > Subsystem: Kingston Technology Company, Inc. Device [2646:5008] > Kernel driver in use: nvme > > IOMMU groups: > IOMMU Group 0: > 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:8a02] > (rev 03) > IOMMU Group 1: > 00:02.0 VGA compatible controller [0300]: Intel Corporation Iris > Plus Graphics G1 (Ice Lake) [8086:8a56] (rev 07) > IOMMU Group 2: > 00:07.0 PCI bridge [0604]: Intel Corporation Ice Lake Thunderbolt 3 > PCI Express Root Port #0 [8086:8a1d] (rev 03) > IOMMU Group 3: > 00:0d.0 USB controller [0c03]: Intel Corporation Ice Lake > Thunderbolt 3 USB Controller [8086:8a13] (rev 03) > 00:0d.2 System peripheral [0880]: Intel Corporation Ice Lake > Thunderbolt 3 NHI #0 [8086:8a17] (rev 03) > IOMMU Group 4: > 00:14.0 USB controller [0c03]: Intel Corporation Ice Lake-LP USB > 3.1 xHCI Host Controller [8086:34ed] (rev 30) > 00:14.2 RAM memory [0500]: Intel Corporation Device [8086:34ef] > (rev 30) > IOMMU Group 5: > 00:14.3 Network controller [0280]: Intel Corporation Killer Wi-Fi 6 > AX1650i 160MHz Wireless Network Adapter (201NGW) [8086:34f0] (rev 30) > IOMMU Group 6: > 00:15.0 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP > Serial IO I2C Controller #0 [8086:34e8] (rev 30) > IOMMU Group 7: > 00:16.0 Communication controller [0780]: Intel Corporation > Management Engine Interface [8086:34e0] (rev 30) > IOMMU Group 8: > 00:17.0 SATA controller [0106]: Intel Corporation Ice Lake-LP SATA > Controller [AHCI mode] [8086:34d3] (rev 30) > IOMMU Group 9: > 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:34be] > (rev 30) > IOMMU Group 10: > 00:1c.7 PCI bridge [0604]: Intel Corporation Device [8086:34bf] > (rev 30) > IOMMU Group 11: > 00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:34b4] > (rev 30) > IOMMU Group 12: > 00:1e.0 Communication controller [0780]: Intel Corporation Ice > Lake-LP Serial IO UART Controller #0 [8086:34a8] (rev 30) > 00:1e.3 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP > Serial IO SPI Controller #1 [8086:34ab] (rev 30) > IOMMU Group 13: > 00:1f.0 ISA bridge [0601]: Intel Corporation Ice Lake-LP LPC > Controller [8086:3482] (rev 30) > 00:1f.3 Audio device [0403]: Intel Corporation Smart Sound > Technology Audio Controller [8086:34c8] (rev 30) > 00:1f.4 SMBus [0c05]: Intel Corporation Ice Lake-LP SMBus > Controller [8086:34a3] (rev 30) > 00:1f.5 Serial bus controller [0c80]: Intel Corporation Ice Lake-LP > SPI Controller [8086:34a4] (rev 30) > IOMMU Group 14: > 01:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 > Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > IOMMU Group 15: > 02:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 > Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > 03:00.0 SATA controller [0106]: ASMedia Technology Inc. Device > [1b21:0625] (rev 01) > IOMMU Group 16: > 02:02.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 > Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > 04:00.0 SATA controller [0106]: ASMedia Technology Inc. Device > [1b21:0625] (rev 01) > IOMMU Group 17: > 02:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 > Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) > IOMMU Group 18: > 2c:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev > 15) > IOMMU Group 19: > 2d:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev > 15) > IOMMU Group 20: > 2e:00.0 Non-Volatile memory controller [0108]: Kingston Technology > Company, Inc. Device [2646:5008] (rev 01) > >
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://listman.redhat.com/mailman/listinfo/vfio-users
