On 08/17/2016 08:00 PM, Laine Stump wrote:
On 08/17/2016 12:13 PM, Andrew Jones wrote:
On Wed, Aug 17, 2016 at 08:08:11PM +0800, Kevin Zhao wrote:
Hi all,
[...]
Hi,
1) If we want to support both PCIe devices and PCI, then things are messy.
Currently we propose dropping PCI support. mach-virt pretty much
exclusively uses virtio, which can be set to PCIe mode (virtio-1.0)
I have a libvirt patch just about ACKed for pushing upstream that will
automatically assign virtio-pci devices to a PCIe slot (if the qemu binary
supports virtio-1.0):
https://www.redhat.com/archives/libvir-list/2016-August/msg00852.html
Separate patches do the same for the e1000e emulated network device (which you
probably don't care about) and the nec-usb-xhci (USB3) controller (more useful):
https://www.redhat.com/archives/libvir-list/2016-August/msg00732.html
Once these are in place, the only type of device of any consequence that I can
see still having no PCIe alternative is audio (even though only the virgl video
device is PCIe, libvirt has always
assigned the primary video to slot 1 on pcie-root anyway (although you
shouldn't put a legacy PCI device on a pcie-root-port or
pcie-switch-downstream-port, it is acceptable to plug it directly into
pcie-root (as long as you know you won't need to hotplug it).
I agree, please don't allow plugging PCI devices into PCIe ports (root ports/downstream
ports). Use pcie.0 bus as you mentioned or start a pci "hierarchy" with a
dmi-pci and pci-pci bridge.
2) root complex ports, switches (upstream/downstream ports) are currently
based on Intel parts. Marcel is thinking about creating generic models.
I say this every time it comes up, so just to be consistent: +1 :-)
I plan do that for my "own selfish" reasons, one of them is to be able to
specify
the IO/MEM range the guest firmware should assign to a PCIe port. But, of
course,
I'll keep them neutral enough; I might need help with testing the mach-virt
machines.
3) libvirt needs to learn how to plug everything together, in proper PCIe
fashion, leaving holes for hotplug.
See above about virtio, although that doesn't cover the whole story. The other
part (which I'm working on right now) is that libvirt needs to automatically
add pcie-root-port,
pcie-switch-upstream-port, and pcie-switch-downstream-port devices as
necessary. With the patches I mentioned above, you still have to manually add
enough pcie-*-port controllers to the config, and
then libvirt will plug the PCIe devices into the right place. This is simple
enough to do, but it does require intervention.
As far as leaving holes for hotplug, there's actually still a bit of an open
question there - with machinetypes that use only legacy-PCI, *all* slots are
hotpluggable, and they're added 31 at a time,
so there was never any question about which slots were hotpluggable, and it
would be very rare to end up with a configuration that had 0 free slots
available for hotplug (actually libvirt would always
make sure there was at least one, but in practice there would be many more open
slots). With PCIe-capable machinetypes that is changed, since the root complex
(pcie-root) doesn't support hotplug, and
new slots are added 1 at a time (pcie-*-port) rather than 31 at a time. This
means you have to really go out of your way if you want open slots for hotplug
(and even if you want devices in the machine
at boot time to be hot-unpluggable).
I agree you need to take into account hotplug for Q35, by leaving some PCIe
root ports available, but it shouldn't be too much trouble.
I'm still not sure just how far we need to go in this regard. We've already
decided that, unless manually set to an address on pcie-root by the
user/management application, all PCI devices will be
auto-assigned to a slot that supports hotplug.
Good idea.
What I'm not sure about is whether we should always auto-add an extra
pcie-*-root to be sure a device can be hotplugged, or if we should admit that 1
available slot isn't good enough for all situations, so we should instead just
leave it up to the user/management to manually add extra ports if they think
they'll want to hotplug something later.
Why not? Leaving 1 or 2 PCIe ports should be enough. On each port you can
hotplug a switch with several downstream ports. You can continue nesting the
switches up to depth of 6-7 I think.
Thanks,
Marcel
[...]
Thanks,
drew
