On 18/06/16 16:42, Alex Williamson wrote:
On Sat, Jun 18, 2016 at 9:11 AM, Samuel Holland <sam...@sholland.org
<mailto:sam...@sholland.org>> wrote:
On 06/17/2016 02:00 PM, Kajim Shanti wrote:
If i decide to replace the mobo, which one would you reccommend as
giving the least problems for devices passthrough?
ASRock generally has a good reputation for not breaking VT-d. Be aware
that any motherboard you buy with multiple PCIe x16 ports will
have them
all in the same IOMMU group.
This is why the continuous recommendations for High End Desktop
Processors and Xeon E5, where this is not the case. Just be aware
that standard Core-i3/5/7 are going to group all the processor root
ports together, which either limits your configurations or requires
the acs override patch. The latter, I don't consider supportable.
There are plenty of usable configurations available for Core
processors, but for more than one discrete GPU, you're going to need
to use PCH-based root ports.
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
I'd actually vote for cheaper variants of skylake hardware for consumer
use as you aren't likely to get the x16 port grouping and there won't be
lots of extra controllers with hardwired groups. The grouping problem
for the x16 ports goes away because these cheaper boards don't offer
sharing of the 16 CPU lanes but instead sub-divide the lanes between
separate bridges as you add cards (i.e. 1 x16 card or 2 x8, etc.). The
intel spec. says the host port will offer full isolation if there is no
inter-device data sharing that doesn't go through the host port which
your root bridges can either guarantee for you .. or in the cheaper case
which you get for free by not letting them share lanes and grouping
exceptions manually. For normal hardware this usually means you didn't
plug the same device into multiple ports - or connect devices with a
cable. ACS does offer optional services to cope even with those cases
but simpler boards will tend to have isolation by that rule and Mine
even has ACS registers that explicitly say it's isolated and give some
optional services I've no idea about.
The exception is if you are running multiple Gen2 legacy devices that
need x16 to hit their maximum throughput or naturally for SLI which is
the only consumer use case I can think of that can hope to even saturate
a single x8 Gen 3 port. SLI devices are clearly in the same IOMMU group
so unless that's your intent don't buy the Z77 type boards.
I have little trouble from running two Vms with a different radeon card
passed to each and the IGD on the host using no ACS isolation patches
and qemu 2.6 / 4.7 kernel as well as a couple of minimal server VMs that
run 24/7 but are low load. The board is an ASUS Z170 Pro (I forget which
variant) with an i5 6600k with the IGD running on the host but no X.
This is all firmly in the upper lower end of consumer grade territory
and is only consistent with minimal VM use but it's great if you don't
need scalability or high availability under all workloads. I don't see
many crashes outside the Win10 VM and those never take other VMs with
them. The IGD crashes the host a lot if you use it for accelerated X -
which is why I don't. I'd turn it off if I could use a less cutting edge
kernel.
For reference .. these are my IOMMU groupings and the relevant devices.
If you want more specific information I can provide working examples
from my hardware. I suspect it may vary a lot between the cheaper boards
as only the first PEG port is in any way standardised.
IOMMU / Device
0 00:00.0 Host bridge [0600]: Intel Corporation Skylake Host
Bridge/DRAM Registers [8086:191f] (rev 07)
1 00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe
Controller (x16) [8086:1901] (rev 07)
2 00:02.0 VGA compatible controller [0300]: Intel Corporation HD
Graphics 530 [8086:1912] (rev 06)
3 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H
USB 3.0 xHCI Controller [8086:a12f] (rev 31)
4 00:16.0 Communication controller [0780]: Intel Corporation Sunrise
Point-H CSME HECI #1 [8086:a13a] (rev 31)
5 00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H
SATA controller [AHCI mode] [8086:a102] (rev 31)
6 00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Root Port #17 [8086:a167] (rev f1)
7 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #1 [8086:a110] (rev f1)
8 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #5 [8086:a114] (rev f1)
9 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #9 [8086:a118] (rev f1)
10 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC
Controller [8086:a145] (rev 31)
10 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H
PMC [8086:a121] (rev 31)
10 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD
Audio [8086:a170] (rev 31)
10 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus
[8086:a123] (rev 31)
10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet
Connection (2) I219-V [8086:15b8] (rev 31)
1 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X] [1002:6798]
1 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
Tahiti XT HDMI Audio [Radeon HD 7970 Series] [1002:aaa0]
12 03:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1142 USB
3.1 Host Controller [1b21:1242]
13 04:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
Inc. [AMD/ATI] Cayman XT [Radeon HD 6970] [1002:6718]
13 04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] [1002:aa80]
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
