On 2025/05/06 17:55, Philippe Mathieu-Daudé wrote:
On 6/5/25 10:12, Michael S. Tsirkin wrote:
On Tue, May 06, 2025 at 09:04:50AM +0100, Daniel P. Berrangé wrote:
On Fri, May 02, 2025 at 03:24:41PM +0200, Philippe Mathieu-Daudé wrote:
Legacy VirtIO devices don't have their endianness clearly defined.
QEMU infers it taking the endianness of the (target) binary, or,
when a target support switching endianness at runtime, taking the
endianness of the vCPU accessing the device.
Devices modelling shouldn't really change depending on a property
of a CPU accessing it.
For heterogeneous systems, it is simpler to break such dev <-> cpu
^^^^^^^^^^^^^
dependency, only allowing generic device models, with no knowledge
of CPU (or DMA controller) accesses.
Therefore we introduce the VIRTIO_LEGACY Kconfig key. We keep the
current default (enabled).
New binaries can set CONFIG_VIRTIO_LEGACY=n to restrict models to
the VirtIO version 1 spec.
IMHO that isn't acceptable. In order to be able to provide an
upgrade path from the old binaries, we need the need the new
binaries to be able to serve the same use cases & disabling
virtio 0.9 support prevents that.
This isn't for the single binary effort, there we are taking a
lot of care to not introduce any change.
This is for after it; once we have a single binary (one architecture
run by an instance) we can start testing heterogeneous emulation
(different architectures in the same instance).
I don't see a compelling
technical reason why we can't support virtio 0.9 from this
endian point.
VirtIO 0.9 needs knowledge of the vCPU architecture to get its
endianness. So we need to somehow propagate that at creation
time, guarantying which vCPU (or set of vCPUs) will access a
virtio device.
The use case I'd like to figure out is how should we model
plugging a virtio 0.9 device in into a fully emulated
ZynqMP machine, which has little-endian ARM cores and big
endian MicroBlaze cores.
Alex said this is unlikely to happen, and better is to
ignore this case by not allowing virtio pre-1.0 devices in
our future heterogeneous emulator.
In this same thread Pierrick pointed me to the reference in
the spec: '2.4.3 Legacy Interfaces: A Note on Virtqueue Endianness',
"It is assumed that the host is already aware of the guest endian."
I suppose this means a pre-1.0 virtio device expect to be used by
a single guest OS, but it is not clear the guest OS as fixed
endianness, because the code path checks vCPU endianness at
runtime, so passing guest endianness as a property to pre-1.0
devices isn't really an option.
Anyway I think 1/ I posted this too early, "speculating" as Pierrick
noticed, and confuse the community w.r.t. "single binary" and
2/ I don' t understand legacy virtio and its endianness handling
enough to figure a clever idea to keep using it heterogeneous
setup, so I'll let this task to someone more familiar with that tech.
It may not be too early as QEMU may already support heterogeneous
endianness.
An Arm CPU have a register to switch the endianness, which depends on
the current Exception level, and QEMU already implements it at least
enough to switch the endianness for virtio if I read the code correctly.
This means QEMU should already be able to handle cases where the
endianness changes depending on:
- timing (due to writes to SCLR or switches among Exception levels) or
- vCPUs (as each vCPU has its own register instance).
If virtio has a problem with heterogeneous endianness, I'm afraid that
it matters even now for Arm emulation.
In my understanding, the virtio code is written carefully enough to
avoid breaking migration and I found no other obvious problems like
crashes and memory leaks with heterogeneous endianness, so I can
conclude that the virtio code requires no change.
However, allowing to have legacy devices with heterogeneous endianness
may still lead to mismatches between QEMU and user's expectations as the
spec is not clear. Deprecating the combination of legacy devices with
CPUs that can switch endianness (and other heterogeneous endianness
scenarios in the future) may be a good idea to avoid that.
Regards,
Akihiko Odaki
