Re: [PATCH 01/17] replay: Fix migration use of clock for statistics

[Pavel Dovgalyuk]

On 23.12.2024 20:26, Peter Xu wrote:

On Sat, Dec 21, 2024 at 01:02:01PM +1000, Nicholas Piggin wrote:

On Sat Dec 21, 2024 at 2:31 AM AEST, Peter Xu wrote:

On Fri, Dec 20, 2024 at 08:42:03PM +1000, Nicholas Piggin wrote:

Migration reads CLOCK_HOST when not holding the replay_mutex, which
asserts when recording a trace. These are not guest visible so should
be CLOCK_REALTIME like other statistics in MigrationState, which do
not require the replay_mutex.


Irrelevant of the change, should we document such lock implications in
timer.h?


I guess the intention was to try to avoid caller caring too much
about replay internals, so I'm not sure if that will help or
hinder understanding :(


CLOCK_HOST should be the wall clock in QEMU, IIUC.  If any QEMU caller
tries to read host wall clock requires some mutex to be held.. then I don't
see how we can avoid mentioning it.  It's indeed weird if we need to take a
feature specific mutex just to read the wallclock.. But maybe I misread the
context somewhere..



I think the big rule is something like "if it affects guest state,
then you must use HOST or VIRTUAL*, if it does not affect guest state


HOST clock logically shouldn't be relevant to guest-state?


CLOCK_HOST is used for rtc by default. As the rtc affects the guest 
state, therefore CLOCK_HOST affects guest state too.


Migration is not related to guest state change, therefore it should 
either use realtime clock, or set some flag to make host clock reads not 
tracked by record/replay.





then you must use REALTIME". record-replay code then takes care of
replay mutex locking.

Does get a little fuzzy around edges in code that is somewhat
aware of record-replay though, like migration/snapshots.


Said that, I agree with the change itself - any measurement may not want to
involve NTP at all... which HOST / gtod will, but REALTIME won't.  However
this patch doesn't seem to be for that purpose..  So I'd like to double
check.

Thanks,

Re: [PATCH v15 15/15] hw/vmapple/vmapple: Add vmapple machine type

[Phil Dennis-Jordan]

On Thu, 19 Dec 2024 at 11:50, Akihiko Odaki 
wrote:

> Reviewing "[PATCH v2 6/6] hw/vmapple: XHCI controller's interrupt
> mapping workaround for macOS", I found a few problems so I'm adding
> comments for them:
>
> On 2024/12/19 5:20, Phil Dennis-Jordan wrote:
> > From: Alexander Graf 
> >
> > Apple defines a new "vmapple" machine type as part of its proprietary
> > macOS Virtualization.Framework vmm. This machine type is similar to the
> > virt one, but with subtle differences in base devices, a few special
> > vmapple device additions and a vastly different boot chain.
> >
> > This patch reimplements this machine type in QEMU. To use it, you
> > have to have a readily installed version of macOS for VMApple,
> > run on macOS with -accel hvf, pass the Virtualization.Framework
> > boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
> > and pass aux and root volume as virtio drives. In addition, you also
> > need to find the machine UUID and pass that as -M vmapple,uuid=
> parameter:
> >
> > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
> >  -bios
> /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
> >  -drive file=aux,if=pflash,format=raw \
> >  -drive file=root,if=pflash,format=raw \
> >  -drive file=aux,if=none,id=aux,format=raw \
> >  -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
> >  -drive file=root,if=none,id=root,format=raw \
> >  -device vmapple-virtio-blk-pci,variant=root,drive=root
> >
> > With all these in place, you should be able to see macOS booting
> > successfully.
> >
> > Known issues:
> >   - Keyboard and mouse/tablet input is laggy. The reason for this is
> > that macOS's XHCI driver seems to expect interrupter mapping to
> > be disabled when MSI/MSI-X is unavailable. I have found a
> > workaround but discovered a bunch of other XHCI spec non-compliance
> > in the process, so I'm fixing all of those in a separate patch
> > set.
> >   - Currently only macOS 12 guests are supported. The boot process for
> > 13+ will need further investigation and adjustment.
> >
> > Signed-off-by: Alexander Graf 
> > Co-authored-by: Phil Dennis-Jordan 
> > Signed-off-by: Phil Dennis-Jordan 
> > Reviewed-by: Akihiko Odaki 
> > Tested-by: Akihiko Odaki 
> > ---
> >
> > v3:
> >   * Rebased on latest upstream, updated affinity and NIC creation
> > API usage
> >   * Included Apple-variant virtio-blk in build dependency
> >   * Updated API usage for setting 'redist-region-count' array-typed
> property on GIC.
> >   * Switched from virtio HID devices (for which macOS 12 does not contain
> > drivers) to an XHCI USB controller and USB HID devices.
> >
> > v4:
> >   * Fixups for v4 changes to the other patches in the set.
> >   * Corrected the assert macro to use
> >   * Removed superfluous endian conversions corresponding to cfg's.
> >   * Init error handling improvement.
> >   * No need to select CPU type on TCG, as only HVF is supported.
> >   * Machine type version bumped to 9.2
> >   * #include order improved
> >
> > v5:
> >   * Fixed memory reservation for ecam alias region.
> >   * Better error handling setting properties on devices.
> >   * Simplified the machine ECID/UUID extraction script and actually
> created a
> > file for it rather than quoting its code in documentation.
> >
> > v7:
> >   * Tiny error handling fix, un-inlined function.
> >
> > v8:
> >   * Use object_property_add_uint64_ptr rather than defining custom UUID
> > property get/set functions.
> >
> > v9:
> >   * Documentation improvements
> >   * Fixed variable name and struct field used during pvpanic device
> creation.
> >
> > v10:
> >   * Documentation fixup for changed virtio-blk device type.
> >   * Small improvements to shell commands in documentation.
> >   * Improved propagation of errors during cfg device instantiation.
> >
> > v11:
> >   * Quoted more strings in the documentation's shell script code.
> >
> > v13:
> >   * Bumped the machine type version from 9.2 to 10.0.
> >
> >   MAINTAINERS |   1 +
> >   contrib/vmapple/uuid.sh |   9 +
> >   docs/system/arm/vmapple.rst |  63 
> >   docs/system/target-arm.rst  |   1 +
> >   hw/vmapple/Kconfig  |  20 ++
> >   hw/vmapple/meson.build  |   1 +
> >   hw/vmapple/vmapple.c| 648 
> >   7 files changed, 743 insertions(+)
> >   create mode 100755 contrib/vmapple/uuid.sh
> >   create mode 100644 docs/system/arm/vmapple.rst
> >   create mode 100644 hw/vmapple/vmapple.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 5d9d65e6df7..a8e3d3b74a2 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -2777,6 +2777,7 @@ M: Phil Dennis-Jordan 
> >   S: Maintained
> >   F: hw/vmapple/*
> >   F: include/hw/vmapple/*
> > +F: docs/system/arm/vmapple.rst
> >
> >   Subsystems
> >   --
> > diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.s

Re: [PATCH v15 05/15] MAINTAINERS: Add myself as maintainer for apple-gfx, reviewer for HVF

[Philippe Mathieu-Daudé]

On 18/12/24 21:20, Phil Dennis-Jordan wrote:

I'm happy to take responsibility for the macOS PV graphics code. As
HVF patches don't seem to get much attention at the moment, I'm also
adding myself as designated reviewer for HVF and x86 HVF to try and
improve that.

I anticipate that the resulting workload should be covered by the
funding I'm receiving for improving Qemu in combination with macOS. As
of right now this runs out at the end of 2024; I expect the workload on
apple-gfx should be relatively minor and manageable in my spare time
beyond that. I may have to remove myself from more general HVF duties
once the contract runs out if it's more than I can manage.

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Roman Bolshakov 
---
  MAINTAINERS | 7 +++
  1 file changed, 7 insertions(+)


Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v15 09/15] gpex: Allow more than 4 legacy IRQs

[Philippe Mathieu-Daudé]

On 18/12/24 21:20, Phil Dennis-Jordan wrote:

From: Alexander Graf 

Some boards such as vmapple don't do real legacy PCI IRQ swizzling.
Instead, they just keep allocating more board IRQ lines for each new
legacy IRQ. Let's support that mode by giving instantiators a new
"nr_irqs" property they can use to support more than 4 legacy IRQ lines.
In this mode, GPEX will export more IRQ lines, one for each device.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v4:

  * Turned pair of IRQ arrays into array of structs.
  * Simplified swizzling logic selection.

v12:

  * Fixed uses of deleted GPEX_NUM_IRQS constant that have been
added to QEMU since this patch was originally written.

  hw/arm/sbsa-ref.c  |  2 +-
  hw/arm/virt.c  |  2 +-
  hw/i386/microvm.c  |  2 +-
  hw/loongarch/virt.c| 12 +--
  hw/mips/loongson3_virt.c   |  2 +-
  hw/openrisc/virt.c | 12 +--
  hw/pci-host/gpex.c | 43 ++
  hw/riscv/virt.c| 12 +--
  hw/xen/xen-pvh-common.c|  2 +-
  hw/xtensa/virt.c   |  2 +-
  include/hw/pci-host/gpex.h |  7 +++
  11 files changed, 61 insertions(+), 37 deletions(-)


Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v15 15/15] hw/vmapple/vmapple: Add vmapple machine type

[Philippe Mathieu-Daudé]

Hi Phil,

On 23/12/24 14:53, Phil Dennis-Jordan wrote:
On Thu, 19 Dec 2024 at 11:50, Akihiko Odaki > wrote:


Reviewing "[PATCH v2 6/6] hw/vmapple: XHCI controller's interrupt
mapping workaround for macOS", I found a few problems so I'm adding
comments for them:

On 2024/12/19 5:20, Phil Dennis-Jordan wrote:
 > From: Alexander Graf mailto:g...@amazon.com>>
 >
 > Apple defines a new "vmapple" machine type as part of its proprietary
 > macOS Virtualization.Framework vmm. This machine type is similar
to the
 > virt one, but with subtle differences in base devices, a few special
 > vmapple device additions and a vastly different boot chain.
 >
 > This patch reimplements this machine type in QEMU. To use it, you
 > have to have a readily installed version of macOS for VMApple,
 > run on macOS with -accel hvf, pass the Virtualization.Framework
 > boot rom (AVPBooter) in via -bios, pass the aux and root volume
as pflash
 > and pass aux and root volume as virtio drives. In addition, you also
 > need to find the machine UUID and pass that as -M vmapple,uuid=
parameter:
 >
 > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
 >      -bios /System/Library/Frameworks/Virtualization.framework/
Versions/A/Resources/AVPBooter.vmapple2.bin
 >      -drive file=aux,if=pflash,format=raw \
 >      -drive file=root,if=pflash,format=raw \
 >      -drive file=aux,if=none,id=aux,format=raw \
 >      -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
 >      -drive file=root,if=none,id=root,format=raw \
 >      -device vmapple-virtio-blk-pci,variant=root,drive=root
 >
 > With all these in place, you should be able to see macOS booting
 > successfully.
 >
 > Known issues:
 >   - Keyboard and mouse/tablet input is laggy. The reason for this is
 >     that macOS's XHCI driver seems to expect interrupter mapping to
 >     be disabled when MSI/MSI-X is unavailable. I have found a
 >     workaround but discovered a bunch of other XHCI spec non-
compliance
 >     in the process, so I'm fixing all of those in a separate patch
 >     set.
 >   - Currently only macOS 12 guests are supported. The boot
process for
 >     13+ will need further investigation and adjustment.
 >
 > Signed-off-by: Alexander Graf mailto:g...@amazon.com>>
 > Co-authored-by: Phil Dennis-Jordan mailto:p...@philjordan.eu>>
 > Signed-off-by: Phil Dennis-Jordan mailto:p...@philjordan.eu>>
 > Reviewed-by: Akihiko Odaki mailto:akihiko.od...@daynix.com>>
 > Tested-by: Akihiko Odaki mailto:akihiko.od...@daynix.com>>
 > ---
 >
 > v3:
 >   * Rebased on latest upstream, updated affinity and NIC creation
 >     API usage
 >   * Included Apple-variant virtio-blk in build dependency
 >   * Updated API usage for setting 'redist-region-count' array-
typed property on GIC.
 >   * Switched from virtio HID devices (for which macOS 12 does not
contain
 >     drivers) to an XHCI USB controller and USB HID devices.
 >
 > v4:
 >   * Fixups for v4 changes to the other patches in the set.
 >   * Corrected the assert macro to use
 >   * Removed superfluous endian conversions corresponding to cfg's.
 >   * Init error handling improvement.
 >   * No need to select CPU type on TCG, as only HVF is supported.
 >   * Machine type version bumped to 9.2
 >   * #include order improved
 >
 > v5:
 >   * Fixed memory reservation for ecam alias region.
 >   * Better error handling setting properties on devices.
 >   * Simplified the machine ECID/UUID extraction script and
actually created a
 >     file for it rather than quoting its code in documentation.
 >
 > v7:
 >   * Tiny error handling fix, un-inlined function.
 >
 > v8:
 >   * Use object_property_add_uint64_ptr rather than defining
custom UUID
 >     property get/set functions.
 >
 > v9:
 >   * Documentation improvements
 >   * Fixed variable name and struct field used during pvpanic
device creation.
 >
 > v10:
 >   * Documentation fixup for changed virtio-blk device type.
 >   * Small improvements to shell commands in documentation.
 >   * Improved propagation of errors during cfg device instantiation.
 >
 > v11:
 >   * Quoted more strings in the documentation's shell script code.
 >
 > v13:
 >   * Bumped the machine type version from 9.2 to 10.0.
 >
 >   MAINTAINERS                 |   1 +
 >   contrib/vmapple/uuid.sh     |   9 +
 >   docs/system/arm/vmapple.rst |  63 
 >   docs/system/target-arm.rst  |   1 +
 >   hw/vmapple/Kconfig          |  20 ++
 >   hw/vmapple/meson.build      |   1 +
 >   hw/vmapple/vmapple.c        | 648 +
 

Re: [PATCH v15 15/15] hw/vmapple/vmapple: Add vmapple machine type

[Phil Dennis-Jordan]

Hi Philippe,

> On 23.12.2024, at 17:36, Philippe Mathieu-Daudé  wrote:
> 
> Hi Phil,
> 
>> On 23/12/24 14:53, Phil Dennis-Jordan wrote:
>> On Thu, 19 Dec 2024 at 11:50, Akihiko Odaki > > wrote:
>>Reviewing "[PATCH v2 6/6] hw/vmapple: XHCI controller's interrupt
>>mapping workaround for macOS", I found a few problems so I'm adding
>>comments for them:
>>On 2024/12/19 5:20, Phil Dennis-Jordan wrote:
>> > From: Alexander Graf mailto:g...@amazon.com>>
>> >
>> > Apple defines a new "vmapple" machine type as part of its proprietary
>> > macOS Virtualization.Framework vmm. This machine type is similar
>>to the
>> > virt one, but with subtle differences in base devices, a few special
>> > vmapple device additions and a vastly different boot chain.
>> >
>> > This patch reimplements this machine type in QEMU. To use it, you
>> > have to have a readily installed version of macOS for VMApple,
>> > run on macOS with -accel hvf, pass the Virtualization.Framework
>> > boot rom (AVPBooter) in via -bios, pass the aux and root volume
>>as pflash
>> > and pass aux and root volume as virtio drives. In addition, you also
>> > need to find the machine UUID and pass that as -M vmapple,uuid=
>>parameter:
>> >
>> > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
>> >  -bios /System/Library/Frameworks/Virtualization.framework/
>>Versions/A/Resources/AVPBooter.vmapple2.bin
>> >  -drive file=aux,if=pflash,format=raw \
>> >  -drive file=root,if=pflash,format=raw \
>> >  -drive file=aux,if=none,id=aux,format=raw \
>> >  -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
>> >  -drive file=root,if=none,id=root,format=raw \
>> >  -device vmapple-virtio-blk-pci,variant=root,drive=root
>> >
>> > With all these in place, you should be able to see macOS booting
>> > successfully.
>> >
>> > Known issues:
>> >   - Keyboard and mouse/tablet input is laggy. The reason for this is
>> > that macOS's XHCI driver seems to expect interrupter mapping to
>> > be disabled when MSI/MSI-X is unavailable. I have found a
>> > workaround but discovered a bunch of other XHCI spec non-
>>compliance
>> > in the process, so I'm fixing all of those in a separate patch
>> > set.
>> >   - Currently only macOS 12 guests are supported. The boot
>>process for
>> > 13+ will need further investigation and adjustment.
>> >
>> > Signed-off-by: Alexander Graf >>
>> > Co-authored-by: Phil Dennis-Jordan >>
>> > Signed-off-by: Phil Dennis-Jordan >>
>> > Reviewed-by: Akihiko Odaki >>
>> > Tested-by: Akihiko Odaki >>
>> > ---
>> >
>> > v3:
>> >   * Rebased on latest upstream, updated affinity and NIC creation
>> > API usage
>> >   * Included Apple-variant virtio-blk in build dependency
>> >   * Updated API usage for setting 'redist-region-count' array-
>>typed property on GIC.
>> >   * Switched from virtio HID devices (for which macOS 12 does not
>>contain
>> > drivers) to an XHCI USB controller and USB HID devices.
>> >
>> > v4:
>> >   * Fixups for v4 changes to the other patches in the set.
>> >   * Corrected the assert macro to use
>> >   * Removed superfluous endian conversions corresponding to cfg's.
>> >   * Init error handling improvement.
>> >   * No need to select CPU type on TCG, as only HVF is supported.
>> >   * Machine type version bumped to 9.2
>> >   * #include order improved
>> >
>> > v5:
>> >   * Fixed memory reservation for ecam alias region.
>> >   * Better error handling setting properties on devices.
>> >   * Simplified the machine ECID/UUID extraction script and
>>actually created a
>> > file for it rather than quoting its code in documentation.
>> >
>> > v7:
>> >   * Tiny error handling fix, un-inlined function.
>> >
>> > v8:
>> >   * Use object_property_add_uint64_ptr rather than defining
>>custom UUID
>> > property get/set functions.
>> >
>> > v9:
>> >   * Documentation improvements
>> >   * Fixed variable name and struct field used during pvpanic
>>device creation.
>> >
>> > v10:
>> >   * Documentation fixup for changed virtio-blk device type.
>> >   * Small improvements to shell commands in documentation.
>> >   * Improved propagation of errors during cfg device instantiation.
>> >
>> > v11:
>> >   * Quoted more strings in the documentation's shell script code.
>> >
>> > v13:
>> >   * Bumped the machine type version from 9.2 to 10.0.
>> >
>> >   MAINTAINERS  

Re: [PATCH 01/17] replay: Fix migration use of clock for statistics

[Peter Xu]

On Sat, Dec 21, 2024 at 01:02:01PM +1000, Nicholas Piggin wrote:
> On Sat Dec 21, 2024 at 2:31 AM AEST, Peter Xu wrote:
> > On Fri, Dec 20, 2024 at 08:42:03PM +1000, Nicholas Piggin wrote:
> > > Migration reads CLOCK_HOST when not holding the replay_mutex, which
> > > asserts when recording a trace. These are not guest visible so should
> > > be CLOCK_REALTIME like other statistics in MigrationState, which do
> > > not require the replay_mutex.
> >
> > Irrelevant of the change, should we document such lock implications in
> > timer.h?
> 
> I guess the intention was to try to avoid caller caring too much
> about replay internals, so I'm not sure if that will help or
> hinder understanding :(

CLOCK_HOST should be the wall clock in QEMU, IIUC.  If any QEMU caller
tries to read host wall clock requires some mutex to be held.. then I don't
see how we can avoid mentioning it.  It's indeed weird if we need to take a
feature specific mutex just to read the wallclock.. But maybe I misread the
context somewhere..

> 
> I think the big rule is something like "if it affects guest state,
> then you must use HOST or VIRTUAL*, if it does not affect guest state

HOST clock logically shouldn't be relevant to guest-state?

> then you must use REALTIME". record-replay code then takes care of
> replay mutex locking.
> 
> Does get a little fuzzy around edges in code that is somewhat
> aware of record-replay though, like migration/snapshots.

Said that, I agree with the change itself - any measurement may not want to
involve NTP at all... which HOST / gtod will, but REALTIME won't.  However
this patch doesn't seem to be for that purpose..  So I'd like to double
check.

Thanks,

-- 
Peter Xu

Re: [PATCH v16 00/14] macOS PV Graphics and new vmapple machine type

[Philippe Mathieu-Daudé]

On 23/12/24 23:16, Phil Dennis-Jordan wrote:

This patch set introduces a new ARM and macOS HVF specific machine type
called "vmapple", as well as a family of display devices based on the
ParavirtualizedGraphics.framework in macOS. One of the display adapter
variants, apple-gfx-mmio, is required for the new machine type, while
apple-gfx-pci can be used to enable 3D graphics acceleration with x86-64
macOS guest OSes.

Previous versions of this patch set were submitted semi-separately:
the original vmapple patch set by Alexander Graf included a monolithic
implementation of apple-gfx-mmio. I subsequently reviewed and reworked
the latter to support the PCI variant of the device as well and submitted
the result in isolation. As requested in subsequent review, I have now
recombined this with the original vmapple patch set, which I have updated
and improved in a few ways as well.

The vmapple machine type approximates the configuration in macOS's own
Virtualization.framework when running arm64 macOS guests. In addition to
generic components such as a GICv3 and an XHCI USB controller, it
includes nonstandard extensions to the virtio block device, a special
"hardware" aes engine, a configuration device, a pvpanic variant, a
"backdoor" interface, and of course the apple-gfx paravirtualised display
adapter.

There are currently a few limitations to this which aren't intrinsic,
just imperfect emulation of the VZF, but it's good enough to be just
about usable for some purposes:

  * macOS 12 guests only. Versions 13+ currently fail during early boot.
  * macOS 11+ arm64 hosts only, with hvf accel. (Perhaps some differences
between Apple M series CPUs and TCG's aarch64 implementation? macOS
hosts only because ParavirtualizedGraphics.framework is a black box
implementing most of the logic behind the apple-gfx device.)
  * PCI devices use legacy IRQs, not MSI/MSI-X. As far as I can tell,
we'd need to include the GICv3 ITS, but it's unclear to me what
exactly needs wiring up.
  * Due to a quirk (bug?) in the macOS XHCI driver when MSI-X is not
available, correct functioning of the USB controller (and thus
keyboard/tablet) requires a small workaround in the XHCI controller
device. This is part of another patch series:
https://patchew.org/QEMU/20241208191646.64857-1-p...@philjordan.eu/
  * The guest OS must first be provisioned using Virtualization.framework;
the disk images can subsequently be used in Qemu. (See docs.)

The apple-gfx device can be used independently from the vmapple machine
type, at least in the PCI variant. It mainly targets x86-64 macOS guests
from version 11 on, but also includes a UEFI bootrom for basic
framebuffer mode. macOS 11 is also required on the host side, as well
as a GPU that supports the Metal API. On the guest side, this provides
3D acceleration/GPGPU support with a baseline Metal feature set,
irrespective of the host GPU's feature set. A few limitations in the
current integration:

  * Although it works fine with TCG, it does not work correctly
cross-architecture: x86-64 guests on arm64 hosts appear to make
some boot progress, but rendering is corrupted. I suspect
incompatible texture memory layouts; I have no idea if this is
fixable.
  * ParavirtualizedGraphics.framework and the guest driver support
multi-headed configurations. The current Qemu integration always
connects precisely 1 display.
  * State serialisation and deserialisation is currently not
implemented, though supported in principle by the framework.
Both apple-gfx variants thus set up a migration blocker.
  * Rendering efficiency could be better. The GPU-rendered guest
framebuffer is copied to system memory and uses Qemu's usual
CPU-based drawing. For maximum efficiency, the Metal texture
containing the guest framebuffer could be drawn directly to
a Metal view in the host window, staying on the GPU. (Similar
to the OpenGL/virgl render path on other platforms.)




v15 -> v16

  * 14 patches now, as patch 8 has already been pulled. (Thanks Philippe!)
  * Fixed a bunch of conflicts with upstream code motion:
 - DEFINE_PROP_END_OF_LIST removal (4/14 - apple-gfx mode list, 7/14 -
   pvpanic-mmio, 10/14 - bdif, 11/14 - cfg device, and
   12/14 - vmapple-virtio-blk)
 - sysemu->system move/rename: (1/14 - ui/qemu-main, 2/14 - apple-gfx,
   9/14 - aes, 10/14 - bdif, 14/14 - vmapple machine type)
  * 14/14 (vmapple machine type):
 - Moved compatibility setting for removing legacy mode from virtio-pci
   to proper global property table rather than (ab)using sugar property.
 - Removed a few superfluous #includes during sysemu rename cleanup.
 - Removed machine type versioning as it's not necessary (yet?)
 - Made memory map array const


Great.


Alexander Graf (8):
   hw: Add vmapple subdir
   hw/misc/pvpanic: Add MMIO interface
   gpex: Allow more than 4 legacy IRQs
   hw/vmapple/aes: Introduce aes engine

Re: [PATCH v15 15/15] hw/vmapple/vmapple: Add vmapple machine type

[Philippe Mathieu-Daudé]

On 23/12/24 23:17, Phil Dennis-Jordan wrote:


 > Do you have other changes in your v16? I'm quite happy to
integrate this
 > v15.

The ones proposed by Akihiko are the only ones. I’ve already
implemented them, so I’m happy to post v16.


OK, rebasing on the upstream master branch has revealed it's not quite 
so simple: there are a bunch of clashes with the recent sysemu->system 
rename and the removal of DEFINE_PROP_END_OF_LIST(). They amount to 


Yep :/

pretty trivial changes, but almost all commits needed adjusting, so 
pulling v15 would be painful. I've just posted my v16 (including all the 
changes discussed here).


Thanks for taking an interest in this patch set!

 >> Adapting this giant macro for each machine type seems rather
error-prone and the kind of thing a computer does better than a
human writing the code. I can't help but wonder if we could define a
generic version in boards.h and only implement the
DEFINE_*_MACHINE{_LATEST,}() wrappers specifically for each machine
type. I've created an issue for this on GitLab: https://gitlab.com/
qemu-project/qemu/-/issues/2744  http://gitlab.com/qemu-project/qemu/-/issues/2744>> I
might attack that once I've cut down my backlog of unmerged patches.
 >
 > Do we really want the vmapple machines to be versioned? I see 3
options:
 > 1/ No (simplest)
 > 2/ Not yet, adding versioning when we see the needs
 > 3/ Yes
 >
 > Personally I prefer/recommend 1/ or 2/ ;)

I didn’t realise these were an option. :-) I was just inheriting
Alex’s code here and assumed it was the standard thing to do.

I could imagine we might change a few things to get better guest
version support and perhaps the guest iCloud support that was added
to the Virtualization.framework in macOS 15. But yeah it seems
unlikely that any of this would cause regressions.

What change would you propose to remove the versioning? Is there a
specific machine type I should use as reference? Or do you just want
to change that yourself, if I push my v16 code to a public repository?


OK, grepping through the code base reveals there are quite a number of 
the "smaller" machine types are unversioned. So I guess that's a no- 
brainer then. I've worked out I essentially just need to move the compat 
properties registration to the main init, the "desc" field on the 
machine class needs to be set, and the type becomes non-abstract. 


Correct.

Everything still appears to work when I add that back in after removing 
all the macro definitions and instantiations. Change included in v16, 
but I've still got the reverse of the removal commit around, so if 
anyone has good arguments why we need it, I can easily add versioning 
back in.


Great :)

Thanks,

Phil.

[PATCH v16 00/14] macOS PV Graphics and new vmapple machine type

[Phil Dennis-Jordan]

This patch set introduces a new ARM and macOS HVF specific machine type
called "vmapple", as well as a family of display devices based on the
ParavirtualizedGraphics.framework in macOS. One of the display adapter
variants, apple-gfx-mmio, is required for the new machine type, while
apple-gfx-pci can be used to enable 3D graphics acceleration with x86-64
macOS guest OSes.

Previous versions of this patch set were submitted semi-separately:
the original vmapple patch set by Alexander Graf included a monolithic
implementation of apple-gfx-mmio. I subsequently reviewed and reworked
the latter to support the PCI variant of the device as well and submitted
the result in isolation. As requested in subsequent review, I have now
recombined this with the original vmapple patch set, which I have updated
and improved in a few ways as well.

The vmapple machine type approximates the configuration in macOS's own
Virtualization.framework when running arm64 macOS guests. In addition to
generic components such as a GICv3 and an XHCI USB controller, it
includes nonstandard extensions to the virtio block device, a special
"hardware" aes engine, a configuration device, a pvpanic variant, a
"backdoor" interface, and of course the apple-gfx paravirtualised display
adapter.

There are currently a few limitations to this which aren't intrinsic,
just imperfect emulation of the VZF, but it's good enough to be just
about usable for some purposes:

 * macOS 12 guests only. Versions 13+ currently fail during early boot.
 * macOS 11+ arm64 hosts only, with hvf accel. (Perhaps some differences
   between Apple M series CPUs and TCG's aarch64 implementation? macOS
   hosts only because ParavirtualizedGraphics.framework is a black box
   implementing most of the logic behind the apple-gfx device.)
 * PCI devices use legacy IRQs, not MSI/MSI-X. As far as I can tell,
   we'd need to include the GICv3 ITS, but it's unclear to me what
   exactly needs wiring up.
 * Due to a quirk (bug?) in the macOS XHCI driver when MSI-X is not
   available, correct functioning of the USB controller (and thus
   keyboard/tablet) requires a small workaround in the XHCI controller
   device. This is part of another patch series:
   https://patchew.org/QEMU/20241208191646.64857-1-p...@philjordan.eu/
 * The guest OS must first be provisioned using Virtualization.framework;
   the disk images can subsequently be used in Qemu. (See docs.)

The apple-gfx device can be used independently from the vmapple machine
type, at least in the PCI variant. It mainly targets x86-64 macOS guests
from version 11 on, but also includes a UEFI bootrom for basic
framebuffer mode. macOS 11 is also required on the host side, as well
as a GPU that supports the Metal API. On the guest side, this provides
3D acceleration/GPGPU support with a baseline Metal feature set,
irrespective of the host GPU's feature set. A few limitations in the
current integration:

 * Although it works fine with TCG, it does not work correctly
   cross-architecture: x86-64 guests on arm64 hosts appear to make
   some boot progress, but rendering is corrupted. I suspect
   incompatible texture memory layouts; I have no idea if this is
   fixable.
 * ParavirtualizedGraphics.framework and the guest driver support
   multi-headed configurations. The current Qemu integration always
   connects precisely 1 display.
 * State serialisation and deserialisation is currently not
   implemented, though supported in principle by the framework.
   Both apple-gfx variants thus set up a migration blocker.
 * Rendering efficiency could be better. The GPU-rendered guest
   framebuffer is copied to system memory and uses Qemu's usual
   CPU-based drawing. For maximum efficiency, the Metal texture
   containing the guest framebuffer could be drawn directly to
   a Metal view in the host window, staying on the GPU. (Similar
   to the OpenGL/virgl render path on other platforms.)

Some of my part of this work has been sponsored by Sauce Labs Inc.

---

v2 -> v3:

 * Merged the apple-gfx and vmapple patchsets.
 * Squashed a bunch of later apple-gfx patches into the main one.
   (dGPU support, queried MMIO area size, host GPU picking logic.)
 * Rebased on latest upstream, fixing any breakages due to internal
   Qemu API changes.
 * apple-gfx: Switched to re-entrant MMIO. This is supported by the
   underlying framework and simplifies the MMIO forwarding code which
   was previously different on x86-64 vs aarch64.
 * vmapple: Fixes for minor bugs and comments from the last round of
   review.
 * vmapple aes, conf, apple-gfx: Switched reset methods to implement
   the ResettableClass base's interface.
 * vmapple: switched from virtio-hid to an XHCI USB controller and
   USB mouse and tablet devices. macOS does not provide drivers for
   virtio HID devices, at least not in version 12's vmapple kernel.
   So input now sort of works (interrupt issues) rather than not
   at all. Use network-based remote access to the guest OS as a
   work-a

[PATCH v16 08/14] gpex: Allow more than 4 legacy IRQs

[Phil Dennis-Jordan]

From: Alexander Graf 

Some boards such as vmapple don't do real legacy PCI IRQ swizzling.
Instead, they just keep allocating more board IRQ lines for each new
legacy IRQ. Let's support that mode by giving instantiators a new
"nr_irqs" property they can use to support more than 4 legacy IRQ lines.
In this mode, GPEX will export more IRQ lines, one for each device.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
Reviewed-by: Philippe Mathieu-Daudé 
---

v4:

 * Turned pair of IRQ arrays into array of structs.
 * Simplified swizzling logic selection.

v12:

 * Fixed uses of deleted GPEX_NUM_IRQS constant that have been
   added to QEMU since this patch was originally written.

 hw/arm/sbsa-ref.c  |  2 +-
 hw/arm/virt.c  |  2 +-
 hw/i386/microvm.c  |  2 +-
 hw/loongarch/virt.c| 12 +--
 hw/mips/loongson3_virt.c   |  2 +-
 hw/openrisc/virt.c | 12 +--
 hw/pci-host/gpex.c | 43 ++
 hw/riscv/virt.c| 12 +--
 hw/xen/xen-pvh-common.c|  2 +-
 hw/xtensa/virt.c   |  2 +-
 include/hw/pci-host/gpex.h |  7 +++
 11 files changed, 61 insertions(+), 37 deletions(-)

diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index 581655d771..6183111f2d 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -673,7 +673,7 @@ static void create_pcie(SBSAMachineState *sms)
 /* Map IO port space */
 sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, base_pio);
 
-for (i = 0; i < GPEX_NUM_IRQS; i++) {
+for (i = 0; i < PCI_NUM_PINS; i++) {
 sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
qdev_get_gpio_in(sms->gic, irq + i));
 gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index f9b3380815..99e0a68b6c 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1547,7 +1547,7 @@ static void create_pcie(VirtMachineState *vms)
 /* Map IO port space */
 sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, base_pio);
 
-for (i = 0; i < GPEX_NUM_IRQS; i++) {
+for (i = 0; i < PCI_NUM_PINS; i++) {
 sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
qdev_get_gpio_in(vms->gic, irq + i));
 gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c
index c3d7fe3c42..a8d354aabe 100644
--- a/hw/i386/microvm.c
+++ b/hw/i386/microvm.c
@@ -139,7 +139,7 @@ static void create_gpex(MicrovmMachineState *mms)
 mms->gpex.mmio64.base, mmio64_alias);
 }
 
-for (i = 0; i < GPEX_NUM_IRQS; i++) {
+for (i = 0; i < PCI_NUM_PINS; i++) {
 sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
x86ms->gsi[mms->gpex.irq + i]);
 }
diff --git a/hw/loongarch/virt.c b/hw/loongarch/virt.c
index 3a905cf71d..1243376588 100644
--- a/hw/loongarch/virt.c
+++ b/hw/loongarch/virt.c
@@ -441,7 +441,7 @@ static void fdt_add_pcie_irq_map_node(const 
LoongArchVirtMachineState *lvms,
 {
 int pin, dev;
 uint32_t irq_map_stride = 0;
-uint32_t full_irq_map[GPEX_NUM_IRQS *GPEX_NUM_IRQS * 10] = {};
+uint32_t full_irq_map[PCI_NUM_PINS * PCI_NUM_PINS * 10] = {};
 uint32_t *irq_map = full_irq_map;
 const MachineState *ms = MACHINE(lvms);
 
@@ -454,11 +454,11 @@ static void fdt_add_pcie_irq_map_node(const 
LoongArchVirtMachineState *lvms,
  * to wrap to any number of devices.
  */
 
-for (dev = 0; dev < GPEX_NUM_IRQS; dev++) {
+for (dev = 0; dev < PCI_NUM_PINS; dev++) {
 int devfn = dev * 0x8;
 
-for (pin = 0; pin  < GPEX_NUM_IRQS; pin++) {
-int irq_nr = 16 + ((pin + PCI_SLOT(devfn)) % GPEX_NUM_IRQS);
+for (pin = 0; pin < PCI_NUM_PINS; pin++) {
+int irq_nr = 16 + ((pin + PCI_SLOT(devfn)) % PCI_NUM_PINS);
 int i = 0;
 
 /* Fill PCI address cells */
@@ -482,7 +482,7 @@ static void fdt_add_pcie_irq_map_node(const 
LoongArchVirtMachineState *lvms,
 
 
 qemu_fdt_setprop(ms->fdt, nodename, "interrupt-map", full_irq_map,
- GPEX_NUM_IRQS * GPEX_NUM_IRQS *
+ PCI_NUM_PINS * PCI_NUM_PINS *
  irq_map_stride * sizeof(uint32_t));
 qemu_fdt_setprop_cells(ms->fdt, nodename, "interrupt-map-mask",
  0x1800, 0, 0, 0x7);
@@ -741,7 +741,7 @@ static void virt_devices_init(DeviceState *pch_pic,
 memory_region_add_subregion(get_system_memory(), VIRT_PCI_IO_BASE,
 pio_alias);
 
-for (i = 0; i < GPEX_NUM_IRQS; i++) {
+for (i = 0; i < PCI_NUM_PINS; i++) {
 sysbus_connect_irq(d, i,
qdev_get_gpio_in(pch_pic, 16 + i));
 gpex_set_irq_num(GPEX_HOST(gpex_dev), i, 16 + i);
diff --git a/hw/mips/loongson3_virt.c b/hw/mips/loongson3_virt.c
index f12f8c3d3c..f3cc7a8376 100644
--- a/hw/mips/loongson3_virt.c
+++ b/hw/mips/

[PATCH v16 14/14] hw/vmapple/vmapple: Add vmapple machine type

[Phil Dennis-Jordan]

From: Alexander Graf 

Apple defines a new "vmapple" machine type as part of its proprietary
macOS Virtualization.Framework vmm. This machine type is similar to the
virt one, but with subtle differences in base devices, a few special
vmapple device additions and a vastly different boot chain.

This patch reimplements this machine type in QEMU. To use it, you
have to have a readily installed version of macOS for VMApple,
run on macOS with -accel hvf, pass the Virtualization.Framework
boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
and pass aux and root volume as virtio drives. In addition, you also
need to find the machine UUID and pass that as -M vmapple,uuid= parameter:

$ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
-bios 
/System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
-drive file=aux,if=pflash,format=raw \
-drive file=root,if=pflash,format=raw \
-drive file=aux,if=none,id=aux,format=raw \
-device vmapple-virtio-blk-pci,variant=aux,drive=aux \
-drive file=root,if=none,id=root,format=raw \
-device vmapple-virtio-blk-pci,variant=root,drive=root

With all these in place, you should be able to see macOS booting
successfully.

Known issues:
 - Keyboard and mouse/tablet input is laggy. The reason for this is
   that macOS's XHCI driver seems to expect interrupter mapping to
   be disabled when MSI/MSI-X is unavailable. I have found a
   workaround but discovered a bunch of other XHCI spec non-compliance
   in the process, so I'm fixing all of those in a separate patch
   set.
 - Currently only macOS 12 guests are supported. The boot process for
   13+ will need further investigation and adjustment.

Signed-off-by: Alexander Graf 
Co-authored-by: Phil Dennis-Jordan 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v3:
 * Rebased on latest upstream, updated affinity and NIC creation
   API usage
 * Included Apple-variant virtio-blk in build dependency
 * Updated API usage for setting 'redist-region-count' array-typed property on 
GIC.
 * Switched from virtio HID devices (for which macOS 12 does not contain
   drivers) to an XHCI USB controller and USB HID devices.

v4:
 * Fixups for v4 changes to the other patches in the set.
 * Corrected the assert macro to use
 * Removed superfluous endian conversions corresponding to cfg's.
 * Init error handling improvement.
 * No need to select CPU type on TCG, as only HVF is supported.
 * Machine type version bumped to 9.2
 * #include order improved

v5:
 * Fixed memory reservation for ecam alias region.
 * Better error handling setting properties on devices.
 * Simplified the machine ECID/UUID extraction script and actually created a
   file for it rather than quoting its code in documentation.

v7:
 * Tiny error handling fix, un-inlined function.

v8:
 * Use object_property_add_uint64_ptr rather than defining custom UUID
   property get/set functions.

v9:
 * Documentation improvements
 * Fixed variable name and struct field used during pvpanic device creation.

v10:
 * Documentation fixup for changed virtio-blk device type.
 * Small improvements to shell commands in documentation.
 * Improved propagation of errors during cfg device instantiation.

v11:
 * Quoted more strings in the documentation's shell script code.

v13:
 * Bumped the machine type version from 9.2 to 10.0.

v16:
 * Fixed #include path breakage from recent sysemu->system path rename.
 * Removed a few superfluous #includes during sysemu rename cleanup.
 * Moved compatibility setting for disabling legacy mode in virtio-pci
   to proper global property table rather than (ab)using sugar property.
 * Removed machine type versioning as it's not necessary (yet?)
 * Made memory map array const


 MAINTAINERS |   1 +
 contrib/vmapple/uuid.sh |   9 +
 docs/system/arm/vmapple.rst |  63 
 docs/system/target-arm.rst  |   1 +
 hw/vmapple/Kconfig  |  20 ++
 hw/vmapple/meson.build  |   1 +
 hw/vmapple/vmapple.c| 612 
 7 files changed, 707 insertions(+)
 create mode 100755 contrib/vmapple/uuid.sh
 create mode 100644 docs/system/arm/vmapple.rst
 create mode 100644 hw/vmapple/vmapple.c

diff --git a/MAINTAINERS b/MAINTAINERS
index e220744abf..3a3d9e19f4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2791,6 +2791,7 @@ M: Phil Dennis-Jordan 
 S: Maintained
 F: hw/vmapple/*
 F: include/hw/vmapple/*
+F: docs/system/arm/vmapple.rst
 
 Subsystems
 --
diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
new file mode 100755
index 00..956e8c3afe
--- /dev/null
+++ b/contrib/vmapple/uuid.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+# Used for converting a guest provisioned using Virtualization.framework
+# for use with the QEMU 'vmapple' aarch64 machine type.
+#
+# Extracts the Machine UUID from Virtualization.framework VM JSON file.
+# (as produced by 'macosvm', passed as comman

[PATCH v16 09/14] hw/vmapple/aes: Introduce aes engine

[Phil Dennis-Jordan]

From: Alexander Graf 

VMApple contains an "aes" engine device that it uses to encrypt and
decrypt its nvram. It has trivial hard coded keys it uses for that
purpose.

Add device emulation for this device model.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v3:

 * Rebased on latest upstream and fixed minor breakages.
 * Replaced legacy device reset method with Resettable method

v4:

 * Improved logging of unimplemented functions and guest errors.
 * Better adherence to naming and coding conventions.
 * Cleaner error handling and recovery, including using g_autoptr

v5:

 * More logging improvements
 * Use xxx64_overflow() functions for hexdump buffer size calculations.

v7:

 * Coding style tweaks.

v8:

 * Further improved logging of guest errors.

v9:

 * Replaced a use of cpu_physical_memory_write with dma_memory_write.
 * Dropped unnecessary use of ternary operator for bool -> 0/1.

v10:

 * Code style and comment improvements.

v16:

 * Fixed #include path breakage from recent sysemu->system path rename.

 hw/vmapple/Kconfig   |   2 +
 hw/vmapple/aes.c | 581 +++
 hw/vmapple/meson.build   |   1 +
 hw/vmapple/trace-events  |  14 +
 include/hw/vmapple/vmapple.h |  17 +
 include/qemu/cutils.h|  15 +
 util/hexdump.c   |  18 ++
 7 files changed, 648 insertions(+)
 create mode 100644 hw/vmapple/aes.c
 create mode 100644 include/hw/vmapple/vmapple.h

diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
index 8b13789179..a73504d599 100644
--- a/hw/vmapple/Kconfig
+++ b/hw/vmapple/Kconfig
@@ -1 +1,3 @@
+config VMAPPLE_AES
+bool
 
diff --git a/hw/vmapple/aes.c b/hw/vmapple/aes.c
new file mode 100644
index 00..3a7641ab4b
--- /dev/null
+++ b/hw/vmapple/aes.c
@@ -0,0 +1,581 @@
+/*
+ * QEMU Apple AES device emulation
+ *
+ * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "trace.h"
+#include "crypto/hash.h"
+#include "crypto/aes.h"
+#include "crypto/cipher.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/vmapple/vmapple.h"
+#include "migration/vmstate.h"
+#include "qemu/cutils.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "system/dma.h"
+
+OBJECT_DECLARE_SIMPLE_TYPE(AESState, APPLE_AES)
+
+#define MAX_FIFO_SIZE 9
+
+#define CMD_KEY   0x1
+#define CMD_KEY_CONTEXT_SHIFT27
+#define CMD_KEY_CONTEXT_MASK (0x1 << CMD_KEY_CONTEXT_SHIFT)
+#define CMD_KEY_SELECT_MAX_IDX   0x7
+#define CMD_KEY_SELECT_SHIFT 24
+#define CMD_KEY_SELECT_MASK  (CMD_KEY_SELECT_MAX_IDX << 
CMD_KEY_SELECT_SHIFT)
+#define CMD_KEY_KEY_LEN_NUM  4u
+#define CMD_KEY_KEY_LEN_SHIFT22
+#define CMD_KEY_KEY_LEN_MASK ((CMD_KEY_KEY_LEN_NUM - 1u) << 
CMD_KEY_KEY_LEN_SHIFT)
+#define CMD_KEY_ENCRYPT_SHIFT20
+#define CMD_KEY_ENCRYPT_MASK (0x1 << CMD_KEY_ENCRYPT_SHIFT)
+#define CMD_KEY_BLOCK_MODE_SHIFT 16
+#define CMD_KEY_BLOCK_MODE_MASK  (0x3 << CMD_KEY_BLOCK_MODE_SHIFT)
+#define CMD_IV0x2
+#define CMD_IV_CONTEXT_SHIFT 26
+#define CMD_IV_CONTEXT_MASK  (0x3 << CMD_KEY_CONTEXT_SHIFT)
+#define CMD_DSB   0x3
+#define CMD_SKG   0x4
+#define CMD_DATA  0x5
+#define CMD_DATA_KEY_CTX_SHIFT   27
+#define CMD_DATA_KEY_CTX_MASK(0x1 << CMD_DATA_KEY_CTX_SHIFT)
+#define CMD_DATA_IV_CTX_SHIFT25
+#define CMD_DATA_IV_CTX_MASK (0x3 << CMD_DATA_IV_CTX_SHIFT)
+#define CMD_DATA_LEN_MASK0xff
+#define CMD_STORE_IV  0x6
+#define CMD_STORE_IV_ADDR_MASK   0xff
+#define CMD_WRITE_REG 0x7
+#define CMD_FLAG  0x8
+#define CMD_FLAG_STOP_MASK   BIT(26)
+#define CMD_FLAG_RAISE_IRQ_MASK  BIT(27)
+#define CMD_FLAG_INFO_MASK   0xff
+#define CMD_MAX   0x10
+
+#define CMD_SHIFT 28
+
+#define REG_STATUS0xc
+#define REG_STATUS_DMA_READ_RUNNING BIT(0)
+#define REG_STATUS_DMA_READ_PENDING BIT(1)
+#define REG_STATUS_DMA_WRITE_RUNNINGBIT(2)
+#define REG_STATUS_DMA_WRITE_PENDINGBIT(3)
+#define REG_STATUS_BUSY BIT(4)
+#define REG_STATUS_EXECUTINGBIT(5)
+#define REG_STATUS_READYBIT(6)
+#define REG_STATUS_TEXT_DPA_SEEDED  BIT(7)
+#define REG_STATUS_UNWRAP_DPA_SEEDEDBIT(8)
+
+#define REG_IRQ_STATUS0x18
+#define REG_IRQ_STATUS_INVALID_CMD  BIT(2)
+#define REG_IRQ_STATUS_FLAG BIT(5)
+#define REG_IRQ_ENABLE0x1c
+#define REG_WATERMARK 0x20
+#define REG_Q_STATUS  0x24
+#define REG_FLAG_INFO 0x30
+#define REG_FIFO  0x200
+
+static const uint32_t key_lens[CMD_KEY_KEY_LEN_NUM] = {
+[0] = 16,
+[1] = 24,
+[2] = 32,
+[3] = 64,
+};
+
+typedef struct Key {
+u

[PATCH v16 04/14] hw/display/apple-gfx: Adds configurable mode list

[Phil Dennis-Jordan]

This change adds a property 'display_modes' on the graphics device
which permits specifying a list of display modes. (screen resolution
and refresh rate)

The property is an array of a custom type to make the syntax slightly
less awkward to use, for example:

-device '{"driver":"apple-gfx-pci", "display-modes":["1920x1080@60", 
"3840x2160@60"]}'

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v4:

 * Switched to the native array property type, which recently gained
 command line support.
 * The property has also been added to the -mmio variant.
 * Tidied up the code a little.

v5:

 * Better error handling and buffer management in property parsing and
   output.

v6:

 * Switched to using NSMutableArray for the mode list to avoid need for
   allocating a temporary array - previously done with alloca.

v7:

 * Simplified error handling in property parsing

v8:

 * More consistent integer variable types.

v9:

 * Re-ordered type definitions so we can drop a 'struct' keyword.

v15:

 * Constified the property table.

v16:

 * Removed the DEFINE_PROP_END_OF_LIST markers to match recent upstream
   changes.

 hw/display/apple-gfx-mmio.m |   7 ++
 hw/display/apple-gfx-pci.m  |   8 ++-
 hw/display/apple-gfx.h  |  11 +++
 hw/display/apple-gfx.m  | 135 +++-
 hw/display/trace-events |   2 +
 5 files changed, 143 insertions(+), 20 deletions(-)

diff --git a/hw/display/apple-gfx-mmio.m b/hw/display/apple-gfx-mmio.m
index 1a46ff48b7..60580a373d 100644
--- a/hw/display/apple-gfx-mmio.m
+++ b/hw/display/apple-gfx-mmio.m
@@ -258,6 +258,11 @@ static void apple_gfx_mmio_reset(Object *obj, ResetType 
type)
 [s->common.pgdev reset];
 }
 
+static const Property apple_gfx_mmio_properties[] = {
+DEFINE_PROP_ARRAY("display-modes", AppleGFXMMIOState,
+  common.num_display_modes, common.display_modes,
+  qdev_prop_display_mode, AppleGFXDisplayMode),
+};
 
 static void apple_gfx_mmio_class_init(ObjectClass *klass, void *data)
 {
@@ -267,6 +272,8 @@ static void apple_gfx_mmio_class_init(ObjectClass *klass, 
void *data)
 rc->phases.hold = apple_gfx_mmio_reset;
 dc->hotpluggable = false;
 dc->realize = apple_gfx_mmio_realize;
+
+device_class_set_props(dc, apple_gfx_mmio_properties);
 }
 
 static TypeInfo apple_gfx_mmio_types[] = {
diff --git a/hw/display/apple-gfx-pci.m b/hw/display/apple-gfx-pci.m
index 5ff6a487cf..4cfc01fc72 100644
--- a/hw/display/apple-gfx-pci.m
+++ b/hw/display/apple-gfx-pci.m
@@ -114,6 +114,12 @@ static void apple_gfx_pci_reset(Object *obj, ResetType 
type)
 [s->common.pgdev reset];
 }
 
+static const Property apple_gfx_pci_properties[] = {
+DEFINE_PROP_ARRAY("display-modes", AppleGFXPCIState,
+  common.num_display_modes, common.display_modes,
+  qdev_prop_display_mode, AppleGFXDisplayMode),
+};
+
 static void apple_gfx_pci_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
@@ -130,7 +136,7 @@ static void apple_gfx_pci_class_init(ObjectClass *klass, 
void *data)
 pci->class_id = PCI_CLASS_DISPLAY_OTHER;
 pci->realize = apple_gfx_pci_realize;
 
-/* TODO: Property for setting mode list */
+device_class_set_props(dc, apple_gfx_pci_properties);
 }
 
 static TypeInfo apple_gfx_pci_types[] = {
diff --git a/hw/display/apple-gfx.h b/hw/display/apple-gfx.h
index ef2455e3bd..a1160bf661 100644
--- a/hw/display/apple-gfx.h
+++ b/hw/display/apple-gfx.h
@@ -16,6 +16,7 @@
 #import 
 #include "qemu/typedefs.h"
 #include "exec/memory.h"
+#include "hw/qdev-properties.h"
 #include "ui/surface.h"
 
 @class PGDeviceDescriptor;
@@ -27,6 +28,12 @@
 
 typedef QTAILQ_HEAD(, PGTask_s) PGTaskList;
 
+typedef struct AppleGFXDisplayMode {
+uint16_t width_px;
+uint16_t height_px;
+uint16_t refresh_rate_hz;
+} AppleGFXDisplayMode;
+
 typedef struct AppleGFXState {
 /* Initialised on init/realize() */
 MemoryRegion iomem_gfx;
@@ -35,6 +42,8 @@ typedef struct AppleGFXState {
 QemuConsole *con;
 id mtl;
 id mtl_queue;
+AppleGFXDisplayMode *display_modes;
+uint32_t num_display_modes;
 
 /* List `tasks` is protected by task_mutex */
 QemuMutex task_mutex;
@@ -62,5 +71,7 @@ void *apple_gfx_host_ptr_for_gpa_range(uint64_t 
guest_physical,
uint64_t length, bool read_only,
MemoryRegion **mapping_in_region);
 
+extern const PropertyInfo qdev_prop_display_mode;
+
 #endif
 
diff --git a/hw/display/apple-gfx.m b/hw/display/apple-gfx.m
index f334ee66fd..b00c72fc4d 100644
--- a/hw/display/apple-gfx.m
+++ b/hw/display/apple-gfx.m
@@ -31,9 +31,10 @@
 #include "system/dma.h"
 #include "ui/console.h"
 
-static const PGDisplayCoord_t apple_gfx_modes[] = {
-{ .x = 1440, .y = 1080 },
-{ .x = 1280, .y = 1024 },
+static const AppleGFXDisplayMode apple_gfx_default_modes[] =

[PATCH v16 05/14] MAINTAINERS: Add myself as maintainer for apple-gfx, reviewer for HVF

[Phil Dennis-Jordan]

I'm happy to take responsibility for the macOS PV graphics code. As
HVF patches don't seem to get much attention at the moment, I'm also
adding myself as designated reviewer for HVF and x86 HVF to try and
improve that.

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Roman Bolshakov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 MAINTAINERS | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 38a290e9c2..2101b51217 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -509,6 +509,7 @@ F: target/arm/hvf/
 X86 HVF CPUs
 M: Cameron Esfahani 
 M: Roman Bolshakov 
+R: Phil Dennis-Jordan 
 W: https://wiki.qemu.org/Features/HVF
 S: Maintained
 F: target/i386/hvf/
@@ -516,6 +517,7 @@ F: target/i386/hvf/
 HVF
 M: Cameron Esfahani 
 M: Roman Bolshakov 
+R: Phil Dennis-Jordan 
 W: https://wiki.qemu.org/Features/HVF
 S: Maintained
 F: accel/hvf/
@@ -2631,6 +2633,11 @@ F: hw/display/edid*
 F: include/hw/display/edid.h
 F: qemu-edid.c
 
+macOS PV Graphics (apple-gfx)
+M: Phil Dennis-Jordan 
+S: Maintained
+F: hw/display/apple-gfx*
+
 PIIX4 South Bridge (i82371AB)
 M: Hervé Poussineau 
 M: Philippe Mathieu-Daudé 
-- 
2.39.5 (Apple Git-154)

[PATCH v16 13/14] hw/block/virtio-blk: Replaces request free function with g_free

[Phil Dennis-Jordan]

The virtio_blk_free_request() function has been a 1-liner forwarding
to g_free() for a while now. We may as well call g_free on the request
pointer directly.

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---
 hw/block/virtio-blk.c  | 43 +++---
 hw/vmapple/virtio-blk.c|  2 +-
 include/hw/virtio/virtio-blk.h |  1 -
 3 files changed, 20 insertions(+), 26 deletions(-)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 22a087529d..f131bd58a2 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -50,11 +50,6 @@ static void virtio_blk_init_request(VirtIOBlock *s, 
VirtQueue *vq,
 req->mr_next = NULL;
 }
 
-void virtio_blk_free_request(VirtIOBlockReq *req)
-{
-g_free(req);
-}
-
 void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status)
 {
 VirtIOBlock *s = req->dev;
@@ -93,7 +88,7 @@ static int virtio_blk_handle_rw_error(VirtIOBlockReq *req, 
int error,
 if (acct_failed) {
 block_acct_failed(blk_get_stats(s->blk), &req->acct);
 }
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 blk_error_action(s->blk, action, is_read, error);
@@ -136,7 +131,7 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
 
 virtio_blk_req_complete(req, VIRTIO_BLK_S_OK);
 block_acct_done(blk_get_stats(s->blk), &req->acct);
-virtio_blk_free_request(req);
+g_free(req);
 }
 }
 
@@ -151,7 +146,7 @@ static void virtio_blk_flush_complete(void *opaque, int ret)
 
 virtio_blk_req_complete(req, VIRTIO_BLK_S_OK);
 block_acct_done(blk_get_stats(s->blk), &req->acct);
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 static void virtio_blk_discard_write_zeroes_complete(void *opaque, int ret)
@@ -169,7 +164,7 @@ static void virtio_blk_discard_write_zeroes_complete(void 
*opaque, int ret)
 if (is_write_zeroes) {
 block_acct_done(blk_get_stats(s->blk), &req->acct);
 }
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 static VirtIOBlockReq *virtio_blk_get_request(VirtIOBlock *s, VirtQueue *vq)
@@ -214,7 +209,7 @@ static void virtio_blk_handle_scsi(VirtIOBlockReq *req)
 
 fail:
 virtio_blk_req_complete(req, status);
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 static inline void submit_requests(VirtIOBlock *s, MultiReqBuffer *mrb,
@@ -612,7 +607,7 @@ static void virtio_blk_zone_report_complete(void *opaque, 
int ret)
 
 out:
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 g_free(data->zone_report_data.zones);
 g_free(data);
 }
@@ -661,7 +656,7 @@ static void virtio_blk_handle_zone_report(VirtIOBlockReq 
*req,
 return;
 out:
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 static void virtio_blk_zone_mgmt_complete(void *opaque, int ret)
@@ -677,7 +672,7 @@ static void virtio_blk_zone_mgmt_complete(void *opaque, int 
ret)
 }
 
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 }
 
 static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
@@ -719,7 +714,7 @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, 
BlockZoneOp op)
 return 0;
 out:
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 return err_status;
 }
 
@@ -750,7 +745,7 @@ static void virtio_blk_zone_append_complete(void *opaque, 
int ret)
 
 out:
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 g_free(data);
 }
 
@@ -788,7 +783,7 @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq 
*req,
 
 out:
 virtio_blk_req_complete(req, err_status);
-virtio_blk_free_request(req);
+g_free(req);
 return err_status;
 }
 
@@ -855,7 +850,7 @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, 
MultiReqBuffer *mrb)
 virtio_blk_req_complete(req, VIRTIO_BLK_S_IOERR);
 block_acct_invalid(blk_get_stats(s->blk),
is_write ? BLOCK_ACCT_WRITE : BLOCK_ACCT_READ);
-virtio_blk_free_request(req);
+g_free(req);
 return 0;
 }
 
@@ -911,7 +906,7 @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, 
MultiReqBuffer *mrb)
   VIRTIO_BLK_ID_BYTES));
 iov_from_buf(in_iov, in_num, 0, serial, size);
 virtio_blk_req_complete(req, VIRTIO_BLK_S_OK);
-virtio_blk_free_request(req);
+g_free(req);
 break;
 }
 case VIRTIO_BLK_T_ZONE_APPEND & ~VIRTIO_BLK_T_OUT:
@@ -943,7 +938,7 @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, 
MultiReqBuffer *mrb)
 if (unlikely(!(type & VIRTIO_BLK_T_OUT) ||
  out_len > sizeof(dwz_hdr))) {
 virtio_blk_req_complete(req, VIRTIO_BLK_S_UNSUPP);
-   

[PATCH v16 12/14] hw/vmapple/virtio-blk: Add support for apple virtio-blk

[Phil Dennis-Jordan]

From: Alexander Graf 

Apple has its own virtio-blk PCI device ID where it deviates from the
official virtio-pci spec slightly: It puts a new "apple type"
field at a static offset in config space and introduces a new barrier
command.

This patch first creates a mechanism for virtio-blk downstream classes to
handle unknown commands. It then creates such a downstream class and a new
vmapple-virtio-blk-pci class which support the additional apple type config
identifier as well as the barrier command.

The 'aux' or 'root' device type are selected using the 'variant' property.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v4:

 * Use recommended object type declaration pattern.
 * Correctly log unimplemented code paths.
 * Most header code moved to .c, type name #defines moved to vmapple.h

v5:

 * Corrected handling of potentially unaligned writes to virtio config area.
 * Simplified passing through device variant type to subobject.

v9:

 * Correctly specify class_size for VMAppleVirtIOBlkClass

v10:

 * Folded v9 patch 16/16 into this one, changing the device type design to
   provide a single device type with a variant property instead of 2 different
   subtypes for aux and root volumes.
 * Tidied up error reporting for the variant property.

v15:

 * Constified the property table.

v16:

 * Removed the DEFINE_PROP_END_OF_LIST marker to match recent upstream
   changes.

 hw/block/virtio-blk.c   |  19 ++-
 hw/core/qdev-properties-system.c|   8 ++
 hw/vmapple/Kconfig  |   3 +
 hw/vmapple/meson.build  |   1 +
 hw/vmapple/virtio-blk.c | 204 
 include/hw/pci/pci_ids.h|   1 +
 include/hw/qdev-properties-system.h |   5 +
 include/hw/virtio/virtio-blk.h  |  12 +-
 include/hw/vmapple/vmapple.h|   2 +
 qapi/virtio.json|  14 ++
 10 files changed, 264 insertions(+), 5 deletions(-)
 create mode 100644 hw/vmapple/virtio-blk.c

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 8806c03f7c..22a087529d 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -50,12 +50,12 @@ static void virtio_blk_init_request(VirtIOBlock *s, 
VirtQueue *vq,
 req->mr_next = NULL;
 }
 
-static void virtio_blk_free_request(VirtIOBlockReq *req)
+void virtio_blk_free_request(VirtIOBlockReq *req)
 {
 g_free(req);
 }
 
-static void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status)
+void virtio_blk_req_complete(VirtIOBlockReq *req, unsigned char status)
 {
 VirtIOBlock *s = req->dev;
 VirtIODevice *vdev = VIRTIO_DEVICE(s);
@@ -966,8 +966,18 @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, 
MultiReqBuffer *mrb)
 break;
 }
 default:
-virtio_blk_req_complete(req, VIRTIO_BLK_S_UNSUPP);
-virtio_blk_free_request(req);
+{
+/*
+ * Give subclasses a chance to handle unknown requests. This way the
+ * class lookup is not in the hot path.
+ */
+VirtIOBlkClass *vbk = VIRTIO_BLK_GET_CLASS(s);
+if (!vbk->handle_unknown_request ||
+!vbk->handle_unknown_request(req, mrb, type)) {
+virtio_blk_req_complete(req, VIRTIO_BLK_S_UNSUPP);
+virtio_blk_free_request(req);
+}
+}
 }
 return 0;
 }
@@ -2043,6 +2053,7 @@ static const TypeInfo virtio_blk_info = {
 .instance_size = sizeof(VirtIOBlock),
 .instance_init = virtio_blk_instance_init,
 .class_init = virtio_blk_class_init,
+.class_size = sizeof(VirtIOBlkClass),
 };
 
 static void virtio_register_types(void)
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index a96675beb0..6be44ddb46 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -1283,3 +1283,11 @@ const PropertyInfo qdev_prop_iothread_vq_mapping_list = {
 .set = set_iothread_vq_mapping_list,
 .release = release_iothread_vq_mapping_list,
 };
+
+const PropertyInfo qdev_prop_vmapple_virtio_blk_variant = {
+.name  = "VMAppleVirtioBlkVariant",
+.enum_table  = &VMAppleVirtioBlkVariant_lookup,
+.get   = qdev_propinfo_get_enum,
+.set   = qdev_propinfo_set_enum,
+.set_default_value = qdev_propinfo_set_default_value_enum,
+};
diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
index 8bbeb9a923..bcd1be63e3 100644
--- a/hw/vmapple/Kconfig
+++ b/hw/vmapple/Kconfig
@@ -7,3 +7,6 @@ config VMAPPLE_BDIF
 config VMAPPLE_CFG
 bool
 
+config VMAPPLE_VIRTIO_BLK
+bool
+
diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
index 64b78693a3..bf17cf906c 100644
--- a/hw/vmapple/meson.build
+++ b/hw/vmapple/meson.build
@@ -1,3 +1,4 @@
 system_ss.add(when: 'CONFIG_VMAPPLE_AES',  if_true: files('aes.c'))
 system_ss.add(when: 'CONFIG_VMAPPLE_BDIF', if_true: files('bdif.c'))
 system_ss.add(when: 'CONFIG_VMAPPLE_CFG',  if_true: files('cfg.c'))
+system_ss.add(w

[PATCH v16 02/14] hw/display/apple-gfx: Introduce ParavirtualizedGraphics.Framework support

[Phil Dennis-Jordan]

MacOS provides a framework (library) that allows any vmm to implement a
paravirtualized 3d graphics passthrough to the host metal stack called
ParavirtualizedGraphics.Framework (PVG). The library abstracts away
almost every aspect of the paravirtualized device model and only provides
and receives callbacks on MMIO access as well as to share memory address
space between the VM and PVG.

This patch implements a QEMU device that drives PVG for the VMApple
variant of it.

Signed-off-by: Alexander Graf 
Co-authored-by: Alexander Graf 

Subsequent changes:

 * Cherry-pick/rebase conflict fixes, API use updates.
 * Moved from hw/vmapple/ (useful outside that machine type)
 * Overhaul of threading model, many thread safety improvements.
 * Asynchronous rendering.
 * Memory and object lifetime fixes.
 * Refactoring to split generic and (vmapple) MMIO variant specific
   code.

Implementation wise, most of the complexity lies in the differing threading
models of ParavirtualizedGraphics.framework, which uses libdispatch and
internal locks, versus QEMU, which heavily uses the BQL, especially during
memory-mapped device I/O. Great care has therefore been taken to prevent
deadlocks by never calling into PVG methods while holding the BQL, and
similarly never acquiring the BQL in a callback from PVG. Different strategies
have been used (libdispatch, blocking and non-blocking BHs, RCU, etc.)
depending on the specific requirements at each framework entry and exit point.

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v2:

 * Cherry-pick/rebase conflict fixes
 * BQL function renaming
 * Moved from hw/vmapple/ (useful outside that machine type)
 * Code review comments: Switched to DEFINE_TYPES macro & little endian
   MMIO.
 * Removed some dead/superfluous code
 * Mad set_mode thread & memory safe
 * Added migration blocker due to lack of (de-)serialisation.
 * Fixes to ObjC refcounting and autorelease pool usage.
 * Fixed ObjC new/init misuse
 * Switched to ObjC category extension for private property.
 * Simplified task memory mapping and made it thread safe.
 * Refactoring to split generic and vmapple MMIO variant specific
   code.
 * Switched to asynchronous MMIO writes on x86-64
 * Rendering and graphics update are now done asynchronously
 * Fixed cursor handling
 * Coding convention fixes
 * Removed software cursor compositing

v3:

 * Rebased on latest upstream, fixed breakages including switching to 
Resettable methods.
 * Squashed patches dealing with dGPUs, MMIO area size, and GPU picking.
 * Allow re-entrant MMIO; this simplifies the code and solves the divergence
   between x86-64 and arm64 variants.

v4:

 * Renamed '-vmapple' device variant to '-mmio'
 * MMIO device type now requires aarch64 host and guest
 * Complete overhaul of the glue code for making Qemu's and
   ParavirtualizedGraphics.framework's threading and synchronisation models
   work together. Calls into PVG are from dispatch queues while the
   BQL-holding initiating thread processes AIO context events; callbacks from
   PVG are scheduled as BHs on the BQL/main AIO context, awaiting completion
   where necessary.
 * Guest frame rendering state is covered by the BQL, with only the PVG calls
   outside the lock, and serialised on the named render_queue.
 * Simplified logic for dropping frames in-flight during mode changes, fixed
   bug in pending frames logic.
 * Addressed smaller code review notes such as: function naming, object type
   declarations, type names/declarations/casts, code formatting, #include
   order, over-cautious ObjC retain/release, what goes in init vs realize,
   etc.

v5:

 * Smaller non-functional fixes in response to review comments, such as using
   NULL for the AIO_WAIT_WHILE context argument, type name formatting,
   deleting leftover debug code, logging improvements, state struct field
   order and documentation improvements, etc.
 * Instead of a single condition variable for all synchronous BH job types,
   there is now one for each callback block. This reduces the number
   of threads being awoken unnecessarily to near zero.
 * MMIO device variant: Unified the BH job for raising interrupts.
 * Use DMA APIs for PVG framework's guest memory read requests.
 * Thread safety improvements: ensure mutable AppleGFXState fields are not
   accessed outside the appropriate lock. Added dedicated mutex for the task
   list.
 * Retain references to MemoryRegions for which there exist mappings in each
   PGTask, and for IOSurface mappings.

v6:

 * Switched PGTask_s's' mapped_regions from GPtrArray to GArray
 * Allow DisplaySurface to manage its own vram now that texture -> vram copy
   occurs under BQL.
 * Memory mapping operations now use RCU_READ_LOCK_GUARD() where possible
   instead of a heavy-weight BH job to acquire the BQL.
 * Changed PVG cursor and mode setting callbacks to kick off BHs instead of
   libdispatch tasks which then locked the BQL explicitly.
 * The single remaining

Re: [PATCH v15 15/15] hw/vmapple/vmapple: Add vmapple machine type

[Phil Dennis-Jordan]

On Mon, 23 Dec 2024 at 18:38, Phil Dennis-Jordan  wrote:

> Hi Philippe,
>
> > On 23.12.2024, at 17:36, Philippe Mathieu-Daudé 
> wrote:
> >
> > Hi Phil,
> >
> >> On 23/12/24 14:53, Phil Dennis-Jordan wrote:
> >> On Thu, 19 Dec 2024 at 11:50, Akihiko Odaki  > wrote:
> >>Reviewing "[PATCH v2 6/6] hw/vmapple: XHCI controller's interrupt
> >>mapping workaround for macOS", I found a few problems so I'm adding
> >>comments for them:
> >>On 2024/12/19 5:20, Phil Dennis-Jordan wrote:
> >> > From: Alexander Graf mailto:g...@amazon.com>>
> >> >
> >> > Apple defines a new "vmapple" machine type as part of its
> proprietary
> >> > macOS Virtualization.Framework vmm. This machine type is similar
> >>to the
> >> > virt one, but with subtle differences in base devices, a few
> special
> >> > vmapple device additions and a vastly different boot chain.
> >> >
> >> > This patch reimplements this machine type in QEMU. To use it, you
> >> > have to have a readily installed version of macOS for VMApple,
> >> > run on macOS with -accel hvf, pass the Virtualization.Framework
> >> > boot rom (AVPBooter) in via -bios, pass the aux and root volume
> >>as pflash
> >> > and pass aux and root volume as virtio drives. In addition, you
> also
> >> > need to find the machine UUID and pass that as -M vmapple,uuid=
> >>parameter:
> >> >
> >> > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
> >> >  -bios /System/Library/Frameworks/Virtualization.framework/
> >>Versions/A/Resources/AVPBooter.vmapple2.bin
> >> >  -drive file=aux,if=pflash,format=raw \
> >> >  -drive file=root,if=pflash,format=raw \
> >> >  -drive file=aux,if=none,id=aux,format=raw \
> >> >  -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
> >> >  -drive file=root,if=none,id=root,format=raw \
> >> >  -device vmapple-virtio-blk-pci,variant=root,drive=root
> >> >
> >> > With all these in place, you should be able to see macOS booting
> >> > successfully.
> >> >
> >> > Known issues:
> >> >   - Keyboard and mouse/tablet input is laggy. The reason for this
> is
> >> > that macOS's XHCI driver seems to expect interrupter mapping
> to
> >> > be disabled when MSI/MSI-X is unavailable. I have found a
> >> > workaround but discovered a bunch of other XHCI spec non-
> >>compliance
> >> > in the process, so I'm fixing all of those in a separate patch
> >> > set.
> >> >   - Currently only macOS 12 guests are supported. The boot
> >>process for
> >> > 13+ will need further investigation and adjustment.
> >> >
> >> > Signed-off-by: Alexander Graf  >>>
> >> > Co-authored-by: Phil Dennis-Jordan  >>>
> >> > Signed-off-by: Phil Dennis-Jordan  >>>
> >> > Reviewed-by: Akihiko Odaki  >>>
> >> > Tested-by: Akihiko Odaki  >>>
> >> > ---
> >> >
> >> > v3:
> >> >   * Rebased on latest upstream, updated affinity and NIC creation
> >> > API usage
> >> >   * Included Apple-variant virtio-blk in build dependency
> >> >   * Updated API usage for setting 'redist-region-count' array-
> >>typed property on GIC.
> >> >   * Switched from virtio HID devices (for which macOS 12 does not
> >>contain
> >> > drivers) to an XHCI USB controller and USB HID devices.
> >> >
> >> > v4:
> >> >   * Fixups for v4 changes to the other patches in the set.
> >> >   * Corrected the assert macro to use
> >> >   * Removed superfluous endian conversions corresponding to cfg's.
> >> >   * Init error handling improvement.
> >> >   * No need to select CPU type on TCG, as only HVF is supported.
> >> >   * Machine type version bumped to 9.2
> >> >   * #include order improved
> >> >
> >> > v5:
> >> >   * Fixed memory reservation for ecam alias region.
> >> >   * Better error handling setting properties on devices.
> >> >   * Simplified the machine ECID/UUID extraction script and
> >>actually created a
> >> > file for it rather than quoting its code in documentation.
> >> >
> >> > v7:
> >> >   * Tiny error handling fix, un-inlined function.
> >> >
> >> > v8:
> >> >   * Use object_property_add_uint64_ptr rather than defining
> >>custom UUID
> >> > property get/set functions.
> >> >
> >> > v9:
> >> >   * Documentation improvements
> >> >   * Fixed variable name and struct field used during pvpanic
> >>device creation.
> >> >
> >> > v10:
> >> >   * Documentation fixup for changed virtio-blk device type.
> >> >   * Small improvements to shell commands in documentation.
> >> >   * I

[PATCH v16 01/14] ui & main loop: Redesign of system-specific main thread event handling

[Phil Dennis-Jordan]

macOS's Cocoa event handling must be done on the initial (main) thread
of the process. Furthermore, if library or application code uses
libdispatch, the main dispatch queue must be handling events on the main
thread as well.

So far, this has affected Qemu in both the Cocoa and SDL UIs, although
in different ways: the Cocoa UI replaces the default qemu_main function
with one that spins Qemu's internal main event loop off onto a
background thread. SDL (which uses Cocoa internally) on the other hand
uses a polling approach within Qemu's main event loop. Events are
polled during the SDL UI's dpy_refresh callback, which happens to run
on the main thread by default.

As UIs are mutually exclusive, this works OK as long as nothing else
needs platform-native event handling. In the next patch, a new device is
introduced based on the ParavirtualizedGraphics.framework in macOS.
This uses libdispatch internally, and only works when events are being
handled on the main runloop. With the current system, it works when
using either the Cocoa or the SDL UI. However, it does not when running
headless. Moreover, any attempt to install a similar scheme to the
Cocoa UI's main thread replacement fails when combined with the SDL
UI.

This change tidies up main thread management to be more flexible.

 * The qemu_main global function pointer is a custom function for the
   main thread, and it may now be NULL. When it is, the main thread
   runs the main Qemu loop. This represents the traditional setup.
 * When non-null, spawning the main Qemu event loop on a separate
   thread is now done centrally rather than inside the Cocoa UI code.
 * For most platforms, qemu_main is indeed NULL by default, but on
   Darwin, it defaults to a function that runs the CFRunLoop.
 * The Cocoa UI sets qemu_main to a function which runs the
   NSApplication event handling runloop, as is usual for a Cocoa app.
 * The SDL UI overrides the qemu_main function to NULL, thus
   specifying that Qemu's main loop must run on the main
   thread.
 * The GTK UI also overrides the qemu_main function to NULL.
 * For other UIs, or in the absence of UIs, the platform's default
   behaviour is followed.

This means that on macOS, the platform's runloop events are always
handled, regardless of chosen UI. The new PV graphics device will
thus work in all configurations. There is no functional change on other
operating systems.

Implementing this via a global function pointer variable is a bit
ugly, but it's probably worth investigating the existing UI thread rule
violations in the SDL (e.g. #2537) and GTK+ back-ends. Fixing those
issues might precipitate requirements similar but not identical to those
of the Cocoa UI; hopefully we'll see some kind of pattern emerge, which
can then be used as a basis for an overhaul. (In fact, it may turn
out to be simplest to split the UI/native platform event thread from the
QEMU main event loop on all platforms, with any UI or even none at all.)

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v5:

 * Simplified the way of setting/clearing the main loop by going back
   to setting qemu_main directly, but narrowing the scope of what it
   needs to do, and it can now be NULL.

v6:

 * Folded function qemu_run_default_main_on_new_thread's code into
   main()
 * Removed whitespace changes left over on lines near code removed
   between v4 and v5

v9:

 * Set qemu_main to NULL for GTK UI as well.

v10:

 * Added comments clarifying the functionality and purpose of qemu_main.

v11:

 * Removed the qemu_main_fn typedef again.
 * Consolidation of main, qemu_default_main, and call_qemu_default_main
   so that the latter has been eliminated altogether.
 * Reinstated the #include  directive, added comment saying
   why it's needed.
 * Improved the comment on the qemu_main global variable.
 * Expanded the commit message.

v12:

 * More precise wording of code comments.

v16:

 * Fixed merge conflict with recent sysemu->system path rename.

 include/qemu-main.h | 14 +++-
 system/main.c   | 37 +++
 ui/cocoa.m  | 54 +++--
 ui/gtk.c|  4 
 ui/sdl2.c   |  4 
 5 files changed, 67 insertions(+), 46 deletions(-)

diff --git a/include/qemu-main.h b/include/qemu-main.h
index 940960a7db..2ee83bedff 100644
--- a/include/qemu-main.h
+++ b/include/qemu-main.h
@@ -5,7 +5,19 @@
 #ifndef QEMU_MAIN_H
 #define QEMU_MAIN_H
 
-int qemu_default_main(void);
+/*
+ * The function to run on the main (initial) thread of the process.
+ * NULL means QEMU's main event loop.
+ * When non-NULL, QEMU's main event loop will run on a purposely created
+ * thread, after which the provided function pointer will be invoked on
+ * the initial thread.
+ * This is useful on platforms which treat the main thread as special
+ * (macOS/Darwin) and/or require all UI API calls to occur from the main
+ * thread. Those platforms can i

[PATCH v16 07/14] hw/misc/pvpanic: Add MMIO interface

[Phil Dennis-Jordan]

From: Alexander Graf 

In addition to the ISA and PCI variants of pvpanic, let's add an MMIO
platform device that we can use in embedded arm environments.

Signed-off-by: Alexander Graf 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v3:
 * Rebased on upstream, updated a header path

v15:
 * Constified the property table.

v16:
 * Removed the DEFINE_PROP_END_OF_LIST marker to match recent upstream
   changes.

 hw/misc/Kconfig   |  4 +++
 hw/misc/meson.build   |  1 +
 hw/misc/pvpanic-mmio.c| 60 +++
 include/hw/misc/pvpanic.h |  1 +
 4 files changed, 66 insertions(+)
 create mode 100644 hw/misc/pvpanic-mmio.c

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 1f1baa5dde..5a6c1603b6 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -145,6 +145,10 @@ config PVPANIC_ISA
 depends on ISA_BUS
 select PVPANIC_COMMON
 
+config PVPANIC_MMIO
+bool
+select PVPANIC_COMMON
+
 config AUX
 bool
 select I2C
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index d02d96e403..4de4db0a60 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -122,6 +122,7 @@ system_ss.add(when: 'CONFIG_ARMSSE_MHU', if_true: 
files('armsse-mhu.c'))
 
 system_ss.add(when: 'CONFIG_PVPANIC_ISA', if_true: files('pvpanic-isa.c'))
 system_ss.add(when: 'CONFIG_PVPANIC_PCI', if_true: files('pvpanic-pci.c'))
+system_ss.add(when: 'CONFIG_PVPANIC_MMIO', if_true: files('pvpanic-mmio.c'))
 system_ss.add(when: 'CONFIG_AUX', if_true: files('auxbus.c'))
 system_ss.add(when: 'CONFIG_ASPEED_SOC', if_true: files(
   'aspeed_hace.c',
diff --git a/hw/misc/pvpanic-mmio.c b/hw/misc/pvpanic-mmio.c
new file mode 100644
index 00..70097cecc7
--- /dev/null
+++ b/hw/misc/pvpanic-mmio.c
@@ -0,0 +1,60 @@
+/*
+ * QEMU simulated pvpanic device (MMIO frontend)
+ *
+ * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/qdev-properties.h"
+#include "hw/misc/pvpanic.h"
+#include "hw/sysbus.h"
+#include "standard-headers/misc/pvpanic.h"
+
+OBJECT_DECLARE_SIMPLE_TYPE(PVPanicMMIOState, PVPANIC_MMIO_DEVICE)
+
+#define PVPANIC_MMIO_SIZE 0x2
+
+struct PVPanicMMIOState {
+SysBusDevice parent_obj;
+
+PVPanicState pvpanic;
+};
+
+static void pvpanic_mmio_initfn(Object *obj)
+{
+PVPanicMMIOState *s = PVPANIC_MMIO_DEVICE(obj);
+
+pvpanic_setup_io(&s->pvpanic, DEVICE(s), PVPANIC_MMIO_SIZE);
+sysbus_init_mmio(SYS_BUS_DEVICE(obj), &s->pvpanic.mr);
+}
+
+static const Property pvpanic_mmio_properties[] = {
+DEFINE_PROP_UINT8("events", PVPanicMMIOState, pvpanic.events,
+  PVPANIC_PANICKED | PVPANIC_CRASH_LOADED),
+};
+
+static void pvpanic_mmio_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+device_class_set_props(dc, pvpanic_mmio_properties);
+set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+}
+
+static const TypeInfo pvpanic_mmio_info = {
+.name  = TYPE_PVPANIC_MMIO_DEVICE,
+.parent= TYPE_SYS_BUS_DEVICE,
+.instance_size = sizeof(PVPanicMMIOState),
+.instance_init = pvpanic_mmio_initfn,
+.class_init= pvpanic_mmio_class_init,
+};
+
+static void pvpanic_register_types(void)
+{
+type_register_static(&pvpanic_mmio_info);
+}
+
+type_init(pvpanic_register_types)
diff --git a/include/hw/misc/pvpanic.h b/include/hw/misc/pvpanic.h
index 9a71a5ad0d..049a94c112 100644
--- a/include/hw/misc/pvpanic.h
+++ b/include/hw/misc/pvpanic.h
@@ -26,6 +26,7 @@
 
 #define TYPE_PVPANIC_ISA_DEVICE "pvpanic"
 #define TYPE_PVPANIC_PCI_DEVICE "pvpanic-pci"
+#define TYPE_PVPANIC_MMIO_DEVICE "pvpanic-mmio"
 
 #define PVPANIC_IOPORT_PROP "ioport"
 
-- 
2.39.5 (Apple Git-154)

[PATCH v16 03/14] hw/display/apple-gfx: Adds PCI implementation

[Phil Dennis-Jordan]

This change wires up the PCI variant of the paravirtualised
graphics device, mainly useful for x86-64 macOS guests, implemented
by macOS's ParavirtualizedGraphics.framework. It builds on code
shared with the vmapple/mmio variant of the PVG device.

Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
---

v4:

 * Threading improvements analogous to those in common apple-gfx code
   and mmio device variant.
 * Smaller code review issues addressed.

v5:

 * Minor error handling improvement.

v6:

 * Removed an unused function parameter.

v9:

 * Fixup of changed common call.
 * Whitespace and comment formatting tweaks.

v11:

 * Comment formatting fix.

 hw/display/Kconfig |   4 +
 hw/display/apple-gfx-pci.m | 150 +
 hw/display/meson.build |   1 +
 3 files changed, 155 insertions(+)
 create mode 100644 hw/display/apple-gfx-pci.m

diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index 6a9b7b19ad..2b53dfd7d2 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -149,3 +149,7 @@ config MAC_PVG_MMIO
 bool
 depends on MAC_PVG && AARCH64
 
+config MAC_PVG_PCI
+bool
+depends on MAC_PVG && PCI
+default y if PCI_DEVICES
diff --git a/hw/display/apple-gfx-pci.m b/hw/display/apple-gfx-pci.m
new file mode 100644
index 00..5ff6a487cf
--- /dev/null
+++ b/hw/display/apple-gfx-pci.m
@@ -0,0 +1,150 @@
+/*
+ * QEMU Apple ParavirtualizedGraphics.framework device, PCI variant
+ *
+ * Copyright © 2023-2024 Phil Dennis-Jordan
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * ParavirtualizedGraphics.framework is a set of libraries that macOS provides
+ * which implements 3d graphics passthrough to the host as well as a
+ * proprietary guest communication channel to drive it. This device model
+ * implements support to drive that library from within QEMU as a PCI device
+ * aimed primarily at x86-64 macOS VMs.
+ */
+
+#include "apple-gfx.h"
+#include "hw/pci/pci_device.h"
+#include "hw/pci/msi.h"
+#include "qapi/error.h"
+#include "trace.h"
+#import 
+
+OBJECT_DECLARE_SIMPLE_TYPE(AppleGFXPCIState, APPLE_GFX_PCI)
+
+struct AppleGFXPCIState {
+PCIDevice parent_obj;
+
+AppleGFXState common;
+};
+
+static const char* apple_gfx_pci_option_rom_path = NULL;
+
+static void apple_gfx_init_option_rom_path(void)
+{
+NSURL *option_rom_url = PGCopyOptionROMURL();
+const char *option_rom_path = option_rom_url.fileSystemRepresentation;
+apple_gfx_pci_option_rom_path = g_strdup(option_rom_path);
+[option_rom_url release];
+}
+
+static void apple_gfx_pci_init(Object *obj)
+{
+AppleGFXPCIState *s = APPLE_GFX_PCI(obj);
+
+if (!apple_gfx_pci_option_rom_path) {
+/*
+ * The following is done on device not class init to avoid running
+ * ObjC code before fork() in -daemonize mode.
+ */
+PCIDeviceClass *pci = PCI_DEVICE_CLASS(object_get_class(obj));
+apple_gfx_init_option_rom_path();
+pci->romfile = apple_gfx_pci_option_rom_path;
+}
+
+apple_gfx_common_init(obj, &s->common, TYPE_APPLE_GFX_PCI);
+}
+
+typedef struct AppleGFXPCIInterruptJob {
+PCIDevice *device;
+uint32_t vector;
+} AppleGFXPCIInterruptJob;
+
+static void apple_gfx_pci_raise_interrupt(void *opaque)
+{
+AppleGFXPCIInterruptJob *job = opaque;
+
+if (msi_enabled(job->device)) {
+msi_notify(job->device, job->vector);
+}
+g_free(job);
+}
+
+static void apple_gfx_pci_interrupt(PCIDevice *dev, uint32_t vector)
+{
+AppleGFXPCIInterruptJob *job;
+
+trace_apple_gfx_raise_irq(vector);
+job = g_malloc0(sizeof(*job));
+job->device = dev;
+job->vector = vector;
+aio_bh_schedule_oneshot(qemu_get_aio_context(),
+apple_gfx_pci_raise_interrupt, job);
+}
+
+static void apple_gfx_pci_realize(PCIDevice *dev, Error **errp)
+{
+AppleGFXPCIState *s = APPLE_GFX_PCI(dev);
+int ret;
+
+pci_register_bar(dev, PG_PCI_BAR_MMIO,
+ PCI_BASE_ADDRESS_SPACE_MEMORY, &s->common.iomem_gfx);
+
+ret = msi_init(dev, 0x0 /* config offset; 0 = find space */,
+   PG_PCI_MAX_MSI_VECTORS, true /* msi64bit */,
+   false /* msi_per_vector_mask */, errp);
+if (ret != 0) {
+return;
+}
+
+@autoreleasepool {
+PGDeviceDescriptor *desc = [PGDeviceDescriptor new];
+desc.raiseInterrupt = ^(uint32_t vector) {
+apple_gfx_pci_interrupt(dev, vector);
+};
+
+apple_gfx_common_realize(&s->common, DEVICE(dev), desc, errp);
+[desc release];
+desc = nil;
+}
+}
+
+static void apple_gfx_pci_reset(Object *obj, ResetType type)
+{
+AppleGFXPCIState *s = APPLE_GFX_PCI(obj);
+[s->common.pgdev reset];
+}
+
+static void apple_gfx_pci_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+PCIDeviceClass *pci = PCI_DEVICE_CLASS(klass);
+ResettableClass *rc = RESETTABLE_CLASS(klass);
+
+   

[PATCH v16 06/14] hw: Add vmapple subdir

[Phil Dennis-Jordan]

From: Alexander Graf 

We will introduce a number of devices that are specific to the vmapple
target machine. To keep them all tidily together, let's put them into
a single target directory.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---
 MAINTAINERS | 7 +++
 hw/Kconfig  | 1 +
 hw/meson.build  | 1 +
 hw/vmapple/Kconfig  | 1 +
 hw/vmapple/meson.build  | 0
 hw/vmapple/trace-events | 2 ++
 hw/vmapple/trace.h  | 1 +
 meson.build | 1 +
 8 files changed, 14 insertions(+)
 create mode 100644 hw/vmapple/Kconfig
 create mode 100644 hw/vmapple/meson.build
 create mode 100644 hw/vmapple/trace-events
 create mode 100644 hw/vmapple/trace.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 2101b51217..e220744abf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2785,6 +2785,13 @@ F: hw/hyperv/hv-balloon*.h
 F: include/hw/hyperv/dynmem-proto.h
 F: include/hw/hyperv/hv-balloon.h
 
+VMapple
+M: Alexander Graf 
+M: Phil Dennis-Jordan 
+S: Maintained
+F: hw/vmapple/*
+F: include/hw/vmapple/*
+
 Subsystems
 --
 Overall Audio backends
diff --git a/hw/Kconfig b/hw/Kconfig
index 1b4e9bb07f..2871784cfd 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -41,6 +41,7 @@ source ufs/Kconfig
 source usb/Kconfig
 source virtio/Kconfig
 source vfio/Kconfig
+source vmapple/Kconfig
 source xen/Kconfig
 source watchdog/Kconfig
 
diff --git a/hw/meson.build b/hw/meson.build
index b827c82c5d..9c4f6d0d63 100644
--- a/hw/meson.build
+++ b/hw/meson.build
@@ -39,6 +39,7 @@ subdir('ufs')
 subdir('usb')
 subdir('vfio')
 subdir('virtio')
+subdir('vmapple')
 subdir('watchdog')
 subdir('xen')
 subdir('xenpv')
diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
new file mode 100644
index 00..8b13789179
--- /dev/null
+++ b/hw/vmapple/Kconfig
@@ -0,0 +1 @@
+
diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
new file mode 100644
index 00..e69de29bb2
diff --git a/hw/vmapple/trace-events b/hw/vmapple/trace-events
new file mode 100644
index 00..9ccc579048
--- /dev/null
+++ b/hw/vmapple/trace-events
@@ -0,0 +1,2 @@
+# See docs/devel/tracing.rst for syntax documentation.
+
diff --git a/hw/vmapple/trace.h b/hw/vmapple/trace.h
new file mode 100644
index 00..572adbefe0
--- /dev/null
+++ b/hw/vmapple/trace.h
@@ -0,0 +1 @@
+#include "trace/trace-hw_vmapple.h"
diff --git a/meson.build b/meson.build
index e62251c7ca..1b77955d46 100644
--- a/meson.build
+++ b/meson.build
@@ -3587,6 +3587,7 @@ if have_system
 'hw/usb',
 'hw/vfio',
 'hw/virtio',
+'hw/vmapple',
 'hw/watchdog',
 'hw/xen',
 'hw/gpio',
-- 
2.39.5 (Apple Git-154)

[PATCH v16 10/14] hw/vmapple/bdif: Introduce vmapple backdoor interface

[Phil Dennis-Jordan]

From: Alexander Graf 

The VMApple machine exposes AUX and ROOT block devices (as well as USB OTG
emulation) via virtio-pci as well as a special, simple backdoor platform
device.

This patch implements this backdoor platform device to the best of my
understanding. I left out any USB OTG parts; they're only needed for
guest recovery and I don't understand the protocol yet.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v4:

 * Moved most header code to .c, rest to vmapple.h
 * Better compliance with coding, naming, and formatting conventions.

v8:

 * Replaced uses of cpu_physical_memory_read with dma_memory_read.
 * Replaced an instance of g_free with g_autofree.

v9:

 * Replaced uses of cpu_physical_memory_write with dma_memory_write.

v15:

 * Constified the property table.

v16:

 * Fixed #include path breakage from recent sysemu->system path rename.
 * Removed the DEFINE_PROP_END_OF_LIST marker to match recent upstream
   changes.

 hw/vmapple/Kconfig   |   3 +
 hw/vmapple/bdif.c| 274 +++
 hw/vmapple/meson.build   |   1 +
 hw/vmapple/trace-events  |   5 +
 include/hw/vmapple/vmapple.h |   2 +
 5 files changed, 285 insertions(+)
 create mode 100644 hw/vmapple/bdif.c

diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
index a73504d599..68f88876eb 100644
--- a/hw/vmapple/Kconfig
+++ b/hw/vmapple/Kconfig
@@ -1,3 +1,6 @@
 config VMAPPLE_AES
 bool
 
+config VMAPPLE_BDIF
+bool
+
diff --git a/hw/vmapple/bdif.c b/hw/vmapple/bdif.c
new file mode 100644
index 00..5827dd2aab
--- /dev/null
+++ b/hw/vmapple/bdif.c
@@ -0,0 +1,274 @@
+/*
+ * VMApple Backdoor Interface
+ *
+ * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "trace.h"
+#include "hw/vmapple/vmapple.h"
+#include "hw/sysbus.h"
+#include "hw/block/block.h"
+#include "qapi/error.h"
+#include "system/block-backend.h"
+#include "system/dma.h"
+
+OBJECT_DECLARE_SIMPLE_TYPE(VMAppleBdifState, VMAPPLE_BDIF)
+
+struct VMAppleBdifState {
+SysBusDevice parent_obj;
+
+BlockBackend *aux;
+BlockBackend *root;
+MemoryRegion mmio;
+};
+
+#define VMAPPLE_BDIF_SIZE   0x0020
+
+#define REG_DEVID_MASK  0x
+#define DEVID_ROOT  0x
+#define DEVID_AUX   0x0001
+#define DEVID_USB   0x0010
+
+#define REG_STATUS  0x0
+#define REG_STATUS_ACTIVE BIT(0)
+#define REG_CFG 0x4
+#define REG_CFG_ACTIVEBIT(1)
+#define REG_UNK10x8
+#define REG_BUSY0x10
+#define REG_BUSY_READYBIT(0)
+#define REG_UNK20x400
+#define REG_CMD 0x408
+#define REG_NEXT_DEVICE 0x420
+#define REG_UNK30x434
+
+typedef struct VblkSector {
+uint32_t pad;
+uint32_t pad2;
+uint32_t sector;
+uint32_t pad3;
+} VblkSector;
+
+typedef struct VblkReqCmd {
+uint64_t addr;
+uint32_t len;
+uint32_t flags;
+} VblkReqCmd;
+
+typedef struct VblkReq {
+VblkReqCmd sector;
+VblkReqCmd data;
+VblkReqCmd retval;
+} VblkReq;
+
+#define VBLK_DATA_FLAGS_READ  0x00030001
+#define VBLK_DATA_FLAGS_WRITE 0x00010001
+
+#define VBLK_RET_SUCCESS  0
+#define VBLK_RET_FAILED   1
+
+static uint64_t bdif_read(void *opaque, hwaddr offset, unsigned size)
+{
+uint64_t ret = -1;
+uint64_t devid = offset & REG_DEVID_MASK;
+
+switch (offset & ~REG_DEVID_MASK) {
+case REG_STATUS:
+ret = REG_STATUS_ACTIVE;
+break;
+case REG_CFG:
+ret = REG_CFG_ACTIVE;
+break;
+case REG_UNK1:
+ret = 0x420;
+break;
+case REG_BUSY:
+ret = REG_BUSY_READY;
+break;
+case REG_UNK2:
+ret = 0x1;
+break;
+case REG_UNK3:
+ret = 0x0;
+break;
+case REG_NEXT_DEVICE:
+switch (devid) {
+case DEVID_ROOT:
+ret = 0x800;
+break;
+case DEVID_AUX:
+ret = 0x1;
+break;
+}
+break;
+}
+
+trace_bdif_read(offset, size, ret);
+return ret;
+}
+
+static void le2cpu_sector(VblkSector *sector)
+{
+sector->sector = le32_to_cpu(sector->sector);
+}
+
+static void le2cpu_reqcmd(VblkReqCmd *cmd)
+{
+cmd->addr = le64_to_cpu(cmd->addr);
+cmd->len = le32_to_cpu(cmd->len);
+cmd->flags = le32_to_cpu(cmd->flags);
+}
+
+static void le2cpu_req(VblkReq *req)
+{
+le2cpu_reqcmd(&req->sector);
+le2cpu_reqcmd(&req->data);
+le2cpu_reqcmd(&req->retval);
+}
+
+static void vblk_cmd(uint64_t devid, BlockBackend *blk, uint64_t gp_addr,
+ uint64_t static_off)
+

[PATCH v16 11/14] hw/vmapple/cfg: Introduce vmapple cfg region

[Phil Dennis-Jordan]

From: Alexander Graf 

Instead of device tree or other more standardized means, VMApple passes
platform configuration to the first stage boot loader in a binary encoded
format that resides at a dedicated RAM region in physical address space.

This patch models this configuration space as a qdev device which we can
then map at the fixed location in the address space. That way, we can
influence and annotate all configuration fields easily.

Signed-off-by: Alexander Graf 
Signed-off-by: Phil Dennis-Jordan 
Reviewed-by: Akihiko Odaki 
Tested-by: Akihiko Odaki 
---

v3:

 * Replaced legacy device reset method with Resettable method

v4:

 * Fixed initialisation of default values for properties
 * Dropped superfluous endianness conversions
 * Moved most header code to .c, device name #define goes in vmapple.h

v5:

 * Improved error reporting in case of string property buffer overflow.

v7:

 * Changed error messages for overrun of properties with
   fixed-length strings to be more useful to users than developers.

v8:

 * Consistent parenthesising of macro arguments for better safety.

v10:

 * Slightly tidier error reporting for overlong property values.

v15:

 * Constified the property table.

v16:

 * Removed the DEFINE_PROP_END_OF_LIST marker to match recent upstream
   changes.

 hw/vmapple/Kconfig   |   3 +
 hw/vmapple/cfg.c | 195 +++
 hw/vmapple/meson.build   |   1 +
 include/hw/vmapple/vmapple.h |   2 +
 4 files changed, 201 insertions(+)
 create mode 100644 hw/vmapple/cfg.c

diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
index 68f88876eb..8bbeb9a923 100644
--- a/hw/vmapple/Kconfig
+++ b/hw/vmapple/Kconfig
@@ -4,3 +4,6 @@ config VMAPPLE_AES
 config VMAPPLE_BDIF
 bool
 
+config VMAPPLE_CFG
+bool
+
diff --git a/hw/vmapple/cfg.c b/hw/vmapple/cfg.c
new file mode 100644
index 00..63414d801f
--- /dev/null
+++ b/hw/vmapple/cfg.c
@@ -0,0 +1,195 @@
+/*
+ * VMApple Configuration Region
+ *
+ * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/vmapple/vmapple.h"
+#include "hw/sysbus.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "qapi/error.h"
+#include "net/net.h"
+
+OBJECT_DECLARE_SIMPLE_TYPE(VMAppleCfgState, VMAPPLE_CFG)
+
+#define VMAPPLE_CFG_SIZE 0x0001
+
+typedef struct VMAppleCfg {
+uint32_t version; /* 0x000 */
+uint32_t nr_cpus; /* 0x004 */
+uint32_t unk1;/* 0x008 */
+uint32_t unk2;/* 0x00c */
+uint32_t unk3;/* 0x010 */
+uint32_t unk4;/* 0x014 */
+uint64_t ecid;/* 0x018 */
+uint64_t ram_size;/* 0x020 */
+uint32_t run_installer1;  /* 0x028 */
+uint32_t unk5;/* 0x02c */
+uint32_t unk6;/* 0x030 */
+uint32_t run_installer2;  /* 0x034 */
+uint32_t rnd; /* 0x038 */
+uint32_t unk7;/* 0x03c */
+MACAddr mac_en0;  /* 0x040 */
+uint8_t pad1[2];
+MACAddr mac_en1;  /* 0x048 */
+uint8_t pad2[2];
+MACAddr mac_wifi0;/* 0x050 */
+uint8_t pad3[2];
+MACAddr mac_bt0;  /* 0x058 */
+uint8_t pad4[2];
+uint8_t reserved[0xa0];   /* 0x060 */
+uint32_t cpu_ids[0x80];   /* 0x100 */
+uint8_t scratch[0x200];   /* 0x180 */
+char serial[32];  /* 0x380 */
+char unk8[32];/* 0x3a0 */
+char model[32];   /* 0x3c0 */
+uint8_t unk9[32]; /* 0x3e0 */
+uint32_t unk10;   /* 0x400 */
+char soc_name[32];/* 0x404 */
+} VMAppleCfg;
+
+struct VMAppleCfgState {
+SysBusDevice parent_obj;
+VMAppleCfg cfg;
+
+MemoryRegion mem;
+char *serial;
+char *model;
+char *soc_name;
+};
+
+static void vmapple_cfg_reset(Object *obj, ResetType type)
+{
+VMAppleCfgState *s = VMAPPLE_CFG(obj);
+VMAppleCfg *cfg;
+
+cfg = memory_region_get_ram_ptr(&s->mem);
+memset(cfg, 0, VMAPPLE_CFG_SIZE);
+*cfg = s->cfg;
+}
+
+static bool set_fixlen_property_or_error(char *restrict dst,
+ const char *restrict src,
+ size_t dst_size, Error **errp,
+ const char *property_name)
+{
+ERRP_GUARD();
+size_t len;
+
+len = g_strlcpy(dst, src, dst_size);
+if (len < dst_size) { /* len does not count nul terminator */
+return true;
+}
+
+error_setg(errp, "Provided value too long for property '%s'", 
property_name);
+error_append_hint(errp, "length (%zu) exceeds maximum of %zu\n",
+  len, dst_size - 1);
+return false;
+}
+
+#define set_fixlen_property_or_return(dst_array, src, errp, property_name) \