On Wed, 25 Oct 2023 09:54:07 +0000
Salil Mehta <salil.me...@huawei.com> wrote:
> > From: David Hildenbrand <da...@redhat.com>
> > Sent: Wednesday, October 25, 2023 10:32 AM
> > To: Salil Mehta <salil.me...@huawei.com>; Igor Mammedov
> > <imamm...@redhat.com>; Salil Mehta <salil.me...@opnsrc.net>
> >
> > On 25.10.23 11:16, Salil Mehta wrote:
> > > Hi Igor,
> > >
> > >> From: Igor Mammedov <imamm...@redhat.com>
> > >> Sent: Tuesday, October 24, 2023 9:06 AM
> > >> To: Salil Mehta <salil.me...@opnsrc.net>
> > >>
> > >> On Wed, 18 Oct 2023 17:48:36 +0100
> > >> Salil Mehta <salil.me...@opnsrc.net> wrote:
> > >>
> > >>> Hi Alex,
> > >>>
> > >>> On 18/10/2023 16:41, Alex Bennée wrote:
> > >>>>
> > >>>> Salil Mehta <salil.me...@opnsrc.net> writes:
> > >>>>
> > >>>>> Hello,
> > >>>>>
> > >>>>> Came across below code excerpt in x86/microvm code and wanted to know
> > >>>>> why 'has_hotpluggable_cpus' flag has been set to 'false' while various
> > >>>>> hot(un)plug APIs have been defined?
> > >>>>>
> > >>>>> static void microvm_class_init(ObjectClass *oc, void *data)
> > >>>>> {
> > >>>>> X86MachineClass *x86mc = X86_MACHINE_CLASS(oc);
> > >>>>> MachineClass *mc = MACHINE_CLASS(oc);
> > >>>>> HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
> > >>>>>
> > >>>>> mc->init = microvm_machine_state_init;
> > >>>>>
> > >>>>> mc->family = "microvm_i386";
> > >>>>> [...]
> > >>>>> mc->max_cpus = 288;
> > >>>>> mc->has_hotpluggable_cpus = false; --------> This one
> > >>>>> [...]
> > >>>>
> > >>>> From the original commit that added it:
> > >>>>
> > >>>> It's a minimalist machine type without PCI nor ACPI support,
> > >>>> designed
> > >>>> for short-lived guests. microvm also establishes a baseline for
> > >>>> benchmarking and optimizing both QEMU and guest operating systems,
> > >>>> since it is optimized for both boot time and footprint.
> > >>>
> > >>>
> > >>> Agreed. It looks like ACPI is supported but neither CPU/Memory Hotplug
> > >>> is supported for this minimalist machine type.
> > >>>
> > >>>
> > >>> static void microvm_devices_init(MicrovmMachineState *mms)
> > >>> {
> > >>> const char *default_firmware;
> > >>> X86MachineState *x86ms = X86_MACHINE(mms);
> > >>>
> > >>> [...]
> > >>>
> > >>> /* Optional and legacy devices */
> > >>> if (x86_machine_is_acpi_enabled(x86ms)) {
> > >>> DeviceState *dev = qdev_new(TYPE_ACPI_GED_X86);
> > >>> qdev_prop_set_uint32(dev, "ged-event", ACPI_GED_PWR_DOWN_EVT);
> > >>> sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, GED_MMIO_BASE);
> > >>> /* sysbus_mmio_map(SYS_BUS_DEVICE(dev), 1, GED_MMIO_BASE_MEMHP);
> > >>>
> > */
> > >>>
> > >>> [...]
> > >>>
> > >>> sysbus_realize(SYS_BUS_DEVICE(dev), &error_fatal);
> > >>> x86ms->acpi_dev = HOTPLUG_HANDLER(dev);
> > >>> }
> > >>> [...]
> > >>> }
> > >>>
> > >>>>
> > >>>> Generally hotplug requires a dance between the VMM and the firmware to
> > >>>> properly shutdown and restart hotplug devices. The principle
> > >>>> communication mechanism for this is ACPI.
> > >>
> > >> firmware part in cpu/mem hoptlug usually provided by QEMU by the way of
> > >> ACPI tables (which may contain AML code that handles dance with QEMU
> > >> while exposing standard interface to guest OS to handle hotplug)
> > >>
> > >>>
> > >>> Agreed.
> > >>>
> > >>>>> /* hotplug (for cpu coldplug) */
> > >>>>> mc->get_hotplug_handler = microvm_get_hotplug_handler;
> > >>>>> hc->pre_plug = microvm_device_pre_plug_cb;
> > >>>>> hc->plug = microvm_device_plug_cb;
> > >>>>> hc->unplug_request = microvm_device_unplug_request_cb;
> > >>>>> hc->unplug = microvm_device_unplug_cb;
> > >>>
> > >>> sorry, I also missed the definitions of the last 2 functions which says
> > >>> that unplug is not supported so perhaps these functions are only
> > >>> required to support cold plugging which corroborates with the comment as
> > >>> well.
> > >>
> > >> this function are usually used for both cold and hotplug of bus-less
> > >> devices.
> > >> They provide an opt-in way for board to wire up such devices (incl.
> > >> CPU).
> > >
> > >
> > > Sure. I understand but microvm does not support hotplug so presence of
> > > unplug{_request} versions brought a doubt in my mind but I realized later
> > > that their definitions were empty. Cold-plug does not require unplug
> > > versions.
> > >
> > > Was there any plan to support hot-plug with microvm in future?
> >
> > At least virtio-mem for memory hotplug should be fairly straight-forward
> > to wire-up I guess. The relation to ACPI are minimal: we currently only
> > use ACPI SRAT to expose the maximum GPA range that e.g., Linux requires
> > early during boot to properly prepare for memory hotplug (size the
> > virtual memory space for the kernel accordingly). One could use
> > alternative (paravirtualized) interfaces for that.
>
> Ok. Light weight VM more in lines of Firecracker to improve boot-up times?
>
> Also, presence of unplug versions gives a wrong impression about code?
unplug handlers could be theoretically used to undo what (pre)plug did
during VM deconstruction. (though it's not used this way today unless
hotplug controller requested it).
What is confusing is interface naming (HotplugHandler) which is not
hotplug limited anymore.
> > The question is whether any form of hotplug is "in the spirit" of microvm.
>
> Understand that. BTW, was it ever made to be used with kata-containers?
>
> Thanks
> Salil.
