How should we expose Virtual GPUs to Nova?
Various discussions have happened on the original spec submission for Mitaka[1]
and the recent submission for Newton[2], however there are a few questions
which need further discussion. But before those question (at the end), some
thinking behind the current plans.
* What sort of model should be use for Virtual GPU provisioning?
Virtual GPUs could be considered to be devices like PCI devices or they could
be considered to be a collection of resources. Some hypervisor implementations
(e.g. XenServer) present pre-defined virtual GPU model types to administrators,
but Hyper-V's remote FX spec[3] is using a resource-based approach for graphics
provisioning.
There is also a lack of consistency between these two approaches in hardware:
Some (Intel's GVT-g) could theoretically support per-GPU configuration, but
this is not supported in other cases (e.g. NVIDIA's vGPU) and not possible in
the case of AMD's MxGPU (which is essentially SR-IOV)
As we want to cope with vendors in the same way, the suggestion is that the
hypervisor/driver should expect to expose discrete pre-defined virtual GPU
model types, rather than using a collection of resources.
* Exposing GPU model types to Nova
Assuming that we have pre-defined model types, how do these get exposed to
Nova? The spec proposed for Mitaka spec suggested a new VGPU resource type,
however qemu's expected implementation of virtualised GPUs ('mediated devices')
is to use VFIO to present PCI-like devices to qemu[4]. In addition, a GPU
device could conceptually be passed through to a single VM or split up and
passed through to multiple VMs, which is very similar to SR-IOV. As such, the
spec proposed Newton suggested re-using the PCI passthrough code.
The issue comes because these devices are not strictly SR-IOV devices, and may
not exist as PCI devices. Therefore exposing these using the PCI address
format introduces complications which either have to be worked around with some
'creative' use of the PCI address format to create the fake PCI devices, or
with refactoring the PCI passthrough code to support multiple addressing types,
effectively making the PCI passthrough code work with any device available on
the host.
We could then have different 'address_type's to represent different types of
resources that were to be made available to VMs.
So, some questions:
How much value is there in genericising the PCI passthrough code to support
multiple address types? What other address types might we want to use - e.g.
USB?, SCSI?, IDE?
How would we represent an address for virtual GPUs? Would we define a format
to be used by each driver, or allow the driver to specify their own address
format? See also virt-device-role-tagging [5] which uses various addresses for
different busses.
The addition of an address_type would likely need a refactor of PCIDevice to be
a HostDevice to minimize confusion - How far should that refactor go? (e.g.
should it include renaming the pci_stats column to host_device _stats?)
I'm sure there are other questions... but I can't think of them now! :)
Bob
[1] https://review.openstack.org/#/c/229351/
[2] https://review.openstack.org/#/c/280099/
[3] https://blueprints.launchpad.net/nova/+spec/hyper-v-remotefx
[4] https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg04201.html
[5]
https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/newton/approved/virt-device-role-tagging.rst
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
