On 7/24/19 2:42 PM, Alex Williamson wrote: > On Wed, 24 Jul 2019 08:43:55 -0600 > Alex Williamson <alex.william...@redhat.com> wrote: > >> On Wed, 24 Jul 2019 18:03:31 +0800 >> Peter Xu <zh...@redhat.com> wrote: >> >>> On Wed, Jul 24, 2019 at 05:39:22AM -0400, Michael S. Tsirkin wrote: >>>> On Wed, Jul 24, 2019 at 03:14:39PM +0800, Peter Xu wrote: >>>>> On Tue, Jul 23, 2019 at 11:26:18AM -0600, Alex Williamson wrote: >>>>>>> On 3/29/19 11:49 AM, Alex Williamson wrote: >>>>>>>> [Cc +Brijesh] >>>>>>>> >>>>>>>> Hi Brijesh, will the change below require the IVRS to be updated to >>>>>>>> include aliases for all BDF ranges behind a conventional bridge? I >>>>>>>> think the Linux code handles this regardless of the firmware provided >>>>>>>> aliases, but is it required per spec for the ACPI tables to include >>>>>>>> bridge aliases? Thanks, > [snip] > > For a data point, I fired up an old 990FX system which includes a > PCIe-to-PCI bridge and I added a plugin PCIe-to-PCI bridge just to keep > things interesting... guess how many alias ranges are in the IVRS... > Yep, just the one built into the motherboard: > > AMD-Vi: Using IVHD type 0x10 > AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300 > AMD-Vi: mmio-addr: 00000000fec30000 > AMD-Vi: DEV_SELECT_RANGE_START devid: 00:00.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 00:00.2 > AMD-Vi: DEV_SELECT devid: 00:09.0 flags: 00 > AMD-Vi: DEV_SELECT devid: 01:00.0 flags: 00 > AMD-Vi: DEV_SELECT devid: 00:0a.0 flags: 00 > AMD-Vi: DEV_SELECT devid: 02:00.0 flags: 00 > AMD-Vi: DEV_SELECT devid: 00:11.0 flags: 00 > AMD-Vi: DEV_SELECT_RANGE_START devid: 00:12.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 00:12.2 > AMD-Vi: DEV_SELECT_RANGE_START devid: 00:13.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 00:13.2 > AMD-Vi: DEV_SELECT devid: 00:14.0 flags: d7 > AMD-Vi: DEV_SELECT devid: 00:14.2 flags: 00 > AMD-Vi: DEV_SELECT devid: 00:14.3 flags: 00 > AMD-Vi: DEV_SELECT devid: 00:14.4 flags: 00 > AMD-Vi: DEV_ALIAS_RANGE devid: 03:00.0 flags: 00 devid_to: > 00:14.4 > AMD-Vi: DEV_RANGE_END devid: 03:1f.7 > > [Everything on bus 03:xx.x is aliased to device 00:14.4, the builtin > PCIe-to-PCI bridge] > > AMD-Vi: DEV_SELECT devid: 00:14.5 flags: 00 > AMD-Vi: DEV_SELECT devid: 00:15.0 flags: 00 > AMD-Vi: DEV_SELECT_RANGE_START devid: 04:00.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 04:1f.7 > AMD-Vi: DEV_SELECT devid: 00:15.1 flags: 00 > AMD-Vi: DEV_SELECT_RANGE_START devid: 05:00.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 05:1f.7 > AMD-Vi: DEV_SELECT devid: 00:15.2 flags: 00 > AMD-Vi: DEV_SELECT_RANGE_START devid: 06:00.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 06:1f.7 > AMD-Vi: DEV_SELECT devid: 00:15.3 flags: 00 > AMD-Vi: DEV_SELECT_RANGE_START devid: 08:00.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 08:1f.7 > AMD-Vi: DEV_SELECT_RANGE_START devid: 00:16.0 flags: 00 > AMD-Vi: DEV_RANGE_END devid: 00:16.2 > AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:14.0 > AMD-Vi: DEV_SPECIAL(HPET[0]) devid: 00:14.0 > AMD-Vi: DEV_SPECIAL(IOAPIC[8]) devid: 00:00.1 > > -[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host > Bridge > +-00.2 Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O > Memory Management Unit (IOMMU) > +-09.0-[01]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host > Controller > +-0a.0-[02]----00.0 Marvell Technology Group Ltd. 88SE9172 SATA > 6Gb/s Controller > +-11.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > SATA Controller [AHCI mode] > +-12.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB OHCI0 Controller > +-12.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB EHCI Controller > +-13.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB OHCI0 Controller > +-13.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB EHCI Controller > +-14.0 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus > Controller > +-14.2 Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia > (Intel HDA) > +-14.3 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > LPC host controller > +-14.4-[03]--+-06.0 NVidia / SGS Thomson (Joint Venture) Riva128 > | \-0e.0 VIA Technologies, Inc. VT6306/7/8 [Fire > II(M)] IEEE 1394 OHCI Controller > +-14.5 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB OHCI2 Controller > +-15.0-[04]----00.0 Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller > +-15.1-[05]----00.0 Etron Technology, Inc. EJ168 USB 3.0 Host > Controller > +-15.2-[06-07]----00.0-[07]----00.0 Realtek Semiconductor Co., > Ltd. Device 8149 > +-15.3-[08]-- > +-16.0 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB OHCI0 Controller > +-16.2 Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 > USB EHCI Controller > +-18.0 Advanced Micro Devices, Inc. [AMD] Family 10h Processor > HyperTransport Configuration > +-18.1 Advanced Micro Devices, Inc. [AMD] Family 10h Processor > Address Map > +-18.2 Advanced Micro Devices, Inc. [AMD] Family 10h Processor > DRAM Controller > +-18.3 Advanced Micro Devices, Inc. [AMD] Family 10h Processor > Miscellaneous Control > \-18.4 Advanced Micro Devices, Inc. [AMD] Family 10h Processor > Link Control > > 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI > Bridge (rev 40) (prog-if 01 [Subtractive decode]) > Bus: primary=00, secondary=03, subordinate=03, sec-latency=64 > > 06:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge > (rev 03) (prog-if 00 [Normal decode]) > Bus: primary=06, secondary=07, subordinate=07, sec-latency=32 > Capabilities: [80] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00 > > I realized as I was writing, that the alias caused by 06:00.0 would be > devfn 00.0 on the secondary bus 07, ie. 07:00.0 would alias to > 07:00.0, so technically there's really not an alias for this small > case. So I replaced the NIC with this: > > +-15.2-[06-07]----00.0-[07]--+-00.0 NEC Corporation OHCI USB > Controller > +-00.1 NEC Corporation OHCI USB > Controller > \-00.2 NEC Corporation uPD72010x > USB 2.0 Controller > > Now functions 07:00.[12] also alias to 07:00.0. The IVRS table is > unaffected. > > I'm tempted to say that QEMU would be no worse than bare metal if we > simply ignore IVHD device alias entries. I know that Linux will figure > out the aliasing regardless of the IVRS. Will Windows guests? I'd > love to hear from Bijesh or Suravee regarding the behavior above versus > what AMD expects from system vendors. Thanks, >
Alex, I am not well versed on the expected IOMMU address space behavior yet, Suravee will know more about. I will ask him to take a look at this thread. thanks > Alex >
