On 06/06/2015 02:45 AM, Laszlo Ersek wrote:
Following up on this cross-posted message, I will send two patch sets,
one for QEMU (to qemu-devel) and another for OVMF (to edk2-devel). With
both in place, OVMF supports multiple PCI root buses.
Hi Laszlo,
Thank you for the patches.
Below I'm writing up the way I tested the feature, plus a few random
notes.
(1) Interrupt line assignments. I patched SeaBIOS (temporarily) to print
interrupt line assignments, and I also patched OVMF (permanently) to
print the same. (See both patches in the OVMF patch
OvmfPkg: PlatformBdsLib: debug log interrupt line assignments
in one of the followup series; the commit message contains the
ad-hoc SeaBIOS patch.) The QEMU command line was:
qemu-system-x86_64 \
-m 2048 \
-M pc \
-enable-kvm \
-device qxl-vga \
\
$FIRMWARE_OPTIONS \
\
-drive id=cdrom,if=none,readonly,format=raw,file=$ISO \
-device virtio-scsi-pci,id=scsi0 \
-device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=0 \
\
-debugcon file:debug.log \
-global isa-debugcon.iobase=0x402 \
\
-monitor stdio \
\
-device pxb,id=bridge1,bus_nr=4 \
\
-netdev user,id=netdev0,hostfwd=tcp:127.0.0.1:2222-:22 \
-device e1000,netdev=netdev0,bus=bridge1,addr=2,romfile= \
\
-netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2223-:22 \
-device e1000,netdev=netdev1,bus=bridge1,addr=3,romfile= \
\
-netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2224-:22 \
-device e1000,netdev=netdev2,bus=bridge1,addr=4,romfile= \
\
-netdev user,id=netdev3,hostfwd=tcp:127.0.0.1:2225-:22 \
-device e1000,netdev=netdev3,bus=bridge1,addr=5,romfile=
The ISO variable pointed to a Fedora 20 LiveCD (although it is not
relevant for this test). FIRMWARE_OPTIONS were
-bios .../out/bios.bin
for the (debug-patched, see above) SeaBIOS test case, and
-drive if=pflash,readonly,format=raw,file=.../OVMF_CODE.fd \
-drive if=pflash,format=raw,file=.../COPY_OF_OVMF_VARS.fd
for the OVMF test case.
As you can see the QEMU command line places four e1000 NICs on one
PXB (I used e1000 because that's what Marcel recommended for all
testing). SeaBIOS then printed
PCI: init bdf=00:01.3 id=8086:7113
assigned irq line 10
PCI: init bdf=00:02.0 id=1b36:0100
assigned irq line 10
PCI: init bdf=00:03.0 id=1af4:1004
assigned irq line 11
PCI: init bdf=04:00.0 id=1b36:0001
assigned irq line 11
PCI: init bdf=05:02.0 id=8086:100e
assigned irq line 10
PCI: init bdf=05:03.0 id=8086:100e
assigned irq line 11
PCI: init bdf=05:04.0 id=8086:100e
assigned irq line 11
PCI: init bdf=05:05.0 id=8086:100e
assigned irq line 10
whereas OVMF printed
SetPciIntLine: PciRoot(0x0)/Pci(0x1,0x3) -> 0x0A
SetPciIntLine: PciRoot(0x0)/Pci(0x2,0x0) -> 0x0A
SetPciIntLine: PciRoot(0x0)/Pci(0x3,0x0) -> 0x0B
SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0) -> 0x0B
SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x2,0x0) -> 0x0A
SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x3,0x0) -> 0x0B
SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x4,0x0) -> 0x0B
SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x5,0x0) -> 0x0A
(Note that the bus number assigned to the devices on the pxb is 5 in
the OVMF case too. This is not visible from the device paths logged
above, but it is clear from edk2's PCI bus driver's log, and the
output of the "PCI" UEFI shell command.)
Therefore I concluded that the IRQ line assignment logic in OVMF
that Gabriel had contributed earlier (and that I updated very
slightly in the OVMF patchset here) continued to work correctly.
(2) Marcel, no new fw_cfg file is necessary for exposing the bus numbers
of the extra root buses to OVMF. OVMF probes all buses and all
devices (function 0) to locate the extra root buses. This procedure
is shortened by adhering to the "etc/extra-pci-roots" fw_cfg file,
which is already there.
Sure. It would not be a problem to add it, but if there is no need for it...
This recommendation came from both Michael & Marcel, and it was
"surprisingly easy" to implement with edk2's PciLib.
(3) We discussed earlier the idea to flip the PXB's _HID and _UID
objects to numeric values in QEMU; I implemented and tested that
(see more test cases below).
(4) Ping test was successful from the OVMF-booted Fedora 20 LivecD
environment, using one e1000 NIC on a PXB.
(5) Ping test was successful from the UEFI shell, using Intel's
proprietary UEFI driver for the e1000 NIC (called PROEFI). The NIC
was located on the same PXB as in the previous point.
(6) Successfully loaded public web pages with Firefox in an OVMF-booted
Windows Server 2012 R2 guest. The NIC was located on the same PXB as
in the previous point.
(7) At least one issue remains to be solved (designed) in QEMU, for both
SeaBIOS's and OVMF's sake: booting off devices that are located on
the PXB. The problem is with the "bootorder" fw_cfg file. Consider
the following example:
/pci@i0cf8/scsi@3/channel@0/disk@0,0
/pci/pci-bridge@0/ethernet@2/ethernet-phy@0
which is generated for the options
-device virtio-scsi-pci,id=scsi0 \
-device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=0 \
\
-device pxb,id=bridge1,bus_nr=4 \
-device e1000,netdev=netdev0,bus=bridge1,addr=2,romfile=,bootindex=1
While the first entry is recognized by both SeaBIOS and OVMF, the
second entry (generated for the NIC hanging off the PXB, see above)
is recognized by neither. (I tested OVMF, and investigated the
SeaBIOS source, for this claim.)
For the SeaBIOS explanation, grep the source code for FW_PCI_DOMAIN.
Thanks for bringing it to my attention.
The OVMF explanation is that OVMF simply rejects the initial
OpenFirmware device path node "/pci" with a controlled parse error
(as opposed to the "/pci@i0cf8" node, which it recognizes and
translates to UEFI in combination with the rest of that OFW device
path).
The "/pci" node comes from QEMU's sysbus_get_fw_dev_path() function,
file "hw/core/sysbus.c", where *neither* of the (s->num_mmio) and
(s->num_pio) branches apply. (The (s->num_pio) branch applies for
the first entry, ie. "/pci@i0cf8".)
Something has to be invented here to clue in both firmwares as to
the root bus number (here bus_nr=4), in a format that is compliant
with the "OpenFirmware unit address" concept. (Note that
"/pci-bridge@0" only gives away the slot number *on* the extra root
bus, not the number of the root bus itself.) For example:
/pci@rootbus4/pci-bridge@0/ethernet@2/ethernet-phy@0
would be acceptable. However, I don't know how to implement this in
sysbus_get_fw_dev_path().
I'll look into it. What is the OpenFirmware unit address" concept ? :)
Thanks,
Marcel
Thanks
Laszlo
