On 09/28/2016 10:10 PM, Alex Williamson wrote: > Oh, thanks for the hint, I did not know this before. I did not want to > ACS-override anyway, and thought there would be mainboard layouts that > would separate those 2 slots even on Z170. Is this not the case? > > > This is not the case. The processor does not support ACS itself and > there is no configuration of processor root ports or downstream switches > that can fix that. On the consumer grade hardware only the PCH-based > root ports can be configured for isolation. Those will often still give > you one or more x16 slots that can host a graphics card, but it is a > longer path to memory and the CPU.
I see. So, can I pretty much assume that every X99 board on the market has two PCIe-slots directly connected to the CPU, and isolated in different IOMMU-groups? > If I try to replicate this, it'll go to sleep but not wake up on the > second command. Maybe that's also because of the passed through ASMedia > Device, I didn't test yet. > > > Ok, that's a start, but I don't even have a lot of confidence in > suspending and resuming the VM with a device assigned. Hmm, could you explain in 3 short sentences why that is? I really would like to deepen my knowledge. Ah, and before I forget, the most important point for me was this (from the first post): 4) When resuming from suspend, the BIOS (I presume, someone with another board from ASRock has the same problems) fucks up TSC, which causes the kernel to turn it off, and the VM to stutter badly afterwards. I have to reboot to get a performing VM. (*2) This is because on a fresh boot, my host decides to use TSC as it's clock source. [ 0.074613] TSC deadline timer enabled [ 1.980617] tsc: Refined TSC clocksource calibration: 4007.999 MHz When I suspend it, and wake it up again, *something* alters the TSC counter, making my kernel reject it from now on: [ 440.964713] TSC synchronization [CPU#0 -> CPU#1]: [ 440.964713] Measured 6260509273 cycles TSC warp between CPUs, turning off TSC clock. [ 440.964715] tsc: Marking TSC unstable due to check_tsc_sync_source failed Then when I start the VM, kvm starts to rightfully complain: [ 463.175389] kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable and the VM (Windows 10) is unusable afterwards, until I reboot the host again. I can even take shutting down the VM before suspend, but I can't take rebooting for every start of the VM. I looked around and only found another person with the same problem on another ASRock board, which makes me think this is something in their bioses. I need to get this solved, and I am even gonna spend money, if it's needed. _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
