On Mon, Aug 12, 2024 at 8:24 PM Daniel P. Berrangé <berra...@redhat.com> wrote: > > On Sat, Aug 10, 2024 at 10:44:59PM +0600, Dorjoy Chowdhury wrote: > > AWS Nitro Enclaves have built-in vhost-vsock device support which > > enables applications in enclave VMs to communicate with the parent > > EC2 VM over vsock. The enclave VMs have dynamic CID while the parent > > always has CID 3. In QEMU, the vsock emulation for nitro enclave is > > added using vhost-user-vsock as opposed to vhost-vsock. vhost-vsock > > doesn't support sibling VM communication which is needed for nitro > > enclaves. > > > > In QEMU's nitro-enclave emulation, for the vsock communication to CID > > 3 to work, another process that does the vsock emulation in userspace > > must be run, for example, vhost-device-vsock[1] from rust-vmm, with > > necessary vsock communication support in another guest VM with CID 3. > > A new mandatory nitro-enclave machine option 'vsock' has been added. > > The value for this option should be the chardev id from the '-chardev' > > option for the vhost-user-vsock device to work. > > > > Using vhost-user-vsock also enables the possibility to implement some > > proxying support in the vhost-user-vsock daemon that will forward all > > the packets to the host machine instead of CID 3 so that users of > > nitro-enclave can run the necessary applications in their host machine > > instead of running another whole VM with CID 3. > > > > [1] https://github.com/rust-vmm/vhost-device/tree/main/vhost-device-vsock > > > > Signed-off-by: Dorjoy Chowdhury <dorjoychy...@gmail.com> > > --- > > backends/hostmem-memfd.c | 2 - > > hw/core/machine.c | 71 +++++++++--------- > > hw/i386/Kconfig | 1 + > > hw/i386/nitro_enclave.c | 123 ++++++++++++++++++++++++++++++++ > > include/hw/boards.h | 2 + > > include/hw/i386/nitro_enclave.h | 8 +++ > > include/sysemu/hostmem.h | 2 + > > 7 files changed, 174 insertions(+), 35 deletions(-) > > > > > diff --git a/hw/i386/nitro_enclave.c b/hw/i386/nitro_enclave.c > > index 98690c6373..280ab4cc9b 100644 > > --- a/hw/i386/nitro_enclave.c > > +++ b/hw/i386/nitro_enclave.c > > @@ -11,11 +11,81 @@ > > #include "qemu/osdep.h" > > #include "qemu/error-report.h" > > #include "qapi/error.h" > > +#include "qom/object_interfaces.h" > > > > +#include "chardev/char.h" > > +#include "hw/sysbus.h" > > #include "hw/core/eif.h" > > #include "hw/i386/x86.h" > > #include "hw/i386/microvm.h" > > #include "hw/i386/nitro_enclave.h" > > +#include "hw/virtio/virtio-mmio.h" > > +#include "hw/virtio/vhost-user-vsock.h" > > +#include "sysemu/hostmem.h" > > + > > +static BusState *find_free_virtio_mmio_bus(void) > > +{ > > + BusChild *kid; > > + BusState *bus = sysbus_get_default(); > > + > > + QTAILQ_FOREACH(kid, &bus->children, sibling) { > > + DeviceState *dev = kid->child; > > + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MMIO)) { > > + VirtIOMMIOProxy *mmio = VIRTIO_MMIO(OBJECT(dev)); > > + VirtioBusState *mmio_virtio_bus = &mmio->bus; > > + BusState *mmio_bus = &mmio_virtio_bus->parent_obj; > > + if (QTAILQ_EMPTY(&mmio_bus->children)) { > > + return mmio_bus; > > + } > > + } > > + } > > + > > + return NULL; > > +} > > + > > +static void vhost_user_vsock_init(NitroEnclaveMachineState *nems) > > +{ > > + DeviceState *dev = qdev_new(TYPE_VHOST_USER_VSOCK); > > + VHostUserVSock *vsock = VHOST_USER_VSOCK(dev); > > + BusState *bus; > > + > > + if (!nems->vsock) { > > + error_report("A valid chardev id for vhost-user-vsock device must > > be " > > + "provided using the 'vsock' machine option"); > > + exit(1); > > + } > > + > > + bus = find_free_virtio_mmio_bus(); > > + if (!bus) { > > + error_report("Failed to find bus for vhost-user-vsock device"); > > + exit(1); > > + } > > + > > + Chardev *chardev = qemu_chr_find(nems->vsock); > > + if (!chardev) { > > + error_report("Failed to find chardev with id %s", nems->vsock); > > + exit(1); > > + } > > + > > + vsock->conf.chardev.chr = chardev; > > + > > + qdev_realize_and_unref(dev, bus, &error_fatal); > > +} > > Why does this machine need to create the vhost-user-vsock device itself ? > Doing it this way prevents the mgmt app from changing any of the other > vsock device settings beyond 'chardev'. The entity creating QEMU can use > -device to create the vsock device. >
Hi Daniel. Good point. The reason to make the vhost-user-vsock device built-in is because nitro VMs will always need it anyway (like the virtio-nsm device which is built-in too). When an EIF image is built using nitro-cli the "init" process in the EIF image tries to connect to (AF_VSOCK, CID 3, port 9000) to send a heartbeat and expects a heartbeat reply. So my understanding is that if we don't create it inside the machine code itself, users would always need to provide the explicit options for the device anyway. But as you point out this also makes the device settings non-configurable. Hey Alex, do you have any suggestions on this? Regards, Dorjoy
