On Wed, 24 Jul 2019 08:43:55 -0600
Alex Williamson <alex.william...@redhat.com> wrote:
> On Wed, 24 Jul 2019 18:03:31 +0800
> Peter Xu <zh...@redhat.com> wrote:
>
> > On Wed, Jul 24, 2019 at 05:39:22AM -0400, Michael S. Tsirkin wrote:
> > > On Wed, Jul 24, 2019 at 03:14:39PM +0800, Peter Xu wrote:
> > > > On Tue, Jul 23, 2019 at 11:26:18AM -0600, Alex Williamson wrote:
> > > > > > On 3/29/19 11:49 AM, Alex Williamson wrote:
> > > > > > > [Cc +Brijesh]
> > > > > > >
> > > > > > > Hi Brijesh, will the change below require the IVRS to be updated
> > > > > > > to
> > > > > > > include aliases for all BDF ranges behind a conventional bridge?
> > > > > > > I
> > > > > > > think the Linux code handles this regardless of the firmware
> > > > > > > provided
> > > > > > > aliases, but is it required per spec for the ACPI tables to
> > > > > > > include
> > > > > > > bridge aliases? Thanks,
[snip]
For a data point, I fired up an old 990FX system which includes a
PCIe-to-PCI bridge and I added a plugin PCIe-to-PCI bridge just to keep
things interesting... guess how many alias ranges are in the IVRS...
Yep, just the one built into the motherboard:
AMD-Vi: Using IVHD type 0x10
AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
AMD-Vi: mmio-addr: 00000000fec30000
AMD-Vi: DEV_SELECT_RANGE_START devid: 00:00.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 00:00.2
AMD-Vi: DEV_SELECT devid: 00:09.0 flags: 00
AMD-Vi: DEV_SELECT devid: 01:00.0 flags: 00
AMD-Vi: DEV_SELECT devid: 00:0a.0 flags: 00
AMD-Vi: DEV_SELECT devid: 02:00.0 flags: 00
AMD-Vi: DEV_SELECT devid: 00:11.0 flags: 00
AMD-Vi: DEV_SELECT_RANGE_START devid: 00:12.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 00:12.2
AMD-Vi: DEV_SELECT_RANGE_START devid: 00:13.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 00:13.2
AMD-Vi: DEV_SELECT devid: 00:14.0 flags: d7
AMD-Vi: DEV_SELECT devid: 00:14.2 flags: 00
AMD-Vi: DEV_SELECT devid: 00:14.3 flags: 00
AMD-Vi: DEV_SELECT devid: 00:14.4 flags: 00
AMD-Vi: DEV_ALIAS_RANGE devid: 03:00.0 flags: 00 devid_to:
00:14.4
AMD-Vi: DEV_RANGE_END devid: 03:1f.7
[Everything on bus 03:xx.x is aliased to device 00:14.4, the builtin
PCIe-to-PCI bridge]
AMD-Vi: DEV_SELECT devid: 00:14.5 flags: 00
AMD-Vi: DEV_SELECT devid: 00:15.0 flags: 00
AMD-Vi: DEV_SELECT_RANGE_START devid: 04:00.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 04:1f.7
AMD-Vi: DEV_SELECT devid: 00:15.1 flags: 00
AMD-Vi: DEV_SELECT_RANGE_START devid: 05:00.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 05:1f.7
AMD-Vi: DEV_SELECT devid: 00:15.2 flags: 00
AMD-Vi: DEV_SELECT_RANGE_START devid: 06:00.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 06:1f.7
AMD-Vi: DEV_SELECT devid: 00:15.3 flags: 00
AMD-Vi: DEV_SELECT_RANGE_START devid: 08:00.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 08:1f.7
AMD-Vi: DEV_SELECT_RANGE_START devid: 00:16.0 flags: 00
AMD-Vi: DEV_RANGE_END devid: 00:16.2
AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:14.0
AMD-Vi: DEV_SPECIAL(HPET[0]) devid: 00:14.0
AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:00.1
-[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host
Bridge
+-00.2 Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O
Memory Management Unit (IOMMU)
+-09.0-[01]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host
Controller
+-0a.0-[02]----00.0 Marvell Technology Group Ltd. 88SE9172 SATA
6Gb/s Controller
+-11.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
SATA Controller [AHCI mode]
+-12.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
OHCI0 Controller
+-12.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
EHCI Controller
+-13.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
OHCI0 Controller
+-13.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
EHCI Controller
+-14.0 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
+-14.2 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel
HDA)
+-14.3 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC
host controller
+-14.4-[03]--+-06.0 NVidia / SGS Thomson (Joint Venture) Riva128
| \-0e.0 VIA Technologies, Inc. VT6306/7/8 [Fire II(M)]
IEEE 1394 OHCI Controller
+-14.5 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
OHCI2 Controller
+-15.0-[04]----00.0 Realtek Semiconductor Co., Ltd.
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
+-15.1-[05]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host
Controller
+-15.2-[06-07]----00.0-[07]----00.0 Realtek Semiconductor Co., Ltd.
Device 8149
+-15.3-[08]--
+-16.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
OHCI0 Controller
+-16.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB
EHCI Controller
+-18.0 Advanced Micro Devices, Inc. [AMD] Family 10h Processor
HyperTransport Configuration
+-18.1 Advanced Micro Devices, Inc. [AMD] Family 10h Processor
Address Map
+-18.2 Advanced Micro Devices, Inc. [AMD] Family 10h Processor DRAM
Controller
+-18.3 Advanced Micro Devices, Inc. [AMD] Family 10h Processor
Miscellaneous Control
\-18.4 Advanced Micro Devices, Inc. [AMD] Family 10h Processor Link
Control
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI
Bridge (rev 40) (prog-if 01 [Subtractive decode])
Bus: primary=00, secondary=03, subordinate=03, sec-latency=64
06:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge
(rev 03) (prog-if 00 [Normal decode])
Bus: primary=06, secondary=07, subordinate=07, sec-latency=32
Capabilities: [80] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00
I realized as I was writing, that the alias caused by 06:00.0 would be
devfn 00.0 on the secondary bus 07, ie. 07:00.0 would alias to
07:00.0, so technically there's really not an alias for this small
case. So I replaced the NIC with this:
+-15.2-[06-07]----00.0-[07]--+-00.0 NEC Corporation OHCI USB
Controller
+-00.1 NEC Corporation OHCI USB
Controller
\-00.2 NEC Corporation uPD72010x USB
2.0 Controller
Now functions 07:00.[12] also alias to 07:00.0. The IVRS table is
unaffected.
I'm tempted to say that QEMU would be no worse than bare metal if we
simply ignore IVHD device alias entries. I know that Linux will figure
out the aliasing regardless of the IVRS. Will Windows guests? I'd
love to hear from Bijesh or Suravee regarding the behavior above versus
what AMD expects from system vendors. Thanks,
Alex
