On Wed, Jan 05, 2022 at 10:38:10AM +0000, Thanos Makatos wrote: > > > > -----Original Message----- > > From: Jag Raman <jag.ra...@oracle.com> > > Sent: 17 December 2021 18:00 > > To: Stefan Hajnoczi <stefa...@redhat.com>; John Levon > > <john.le...@nutanix.com>; Thanos Makatos <thanos.maka...@nutanix.com> > > Cc: qemu-devel <qemu-devel@nongnu.org>; Alex Williamson > > <alex.william...@redhat.com>; Marc-André Lureau > > <marcandre.lur...@gmail.com>; Philippe Mathieu-Daudé > > <phi...@redhat.com>; pbonz...@redhat.com; alex.ben...@linaro.org; > > th...@redhat.com; cr...@redhat.com; waine...@redhat.com; > > bl...@redhat.com; Elena Ufimtseva <elena.ufimts...@oracle.com>; John > > Levon <john.le...@nutanix.com>; John Johnson > > <john.g.john...@oracle.com>; Thanos Makatos > > <thanos.maka...@nutanix.com>; Swapnil Ingle <swapnil.in...@nutanix.com> > > Subject: Re: [PATCH v4 07/14] vfio-user: run vfio-user context > > > > > > > > > On Dec 16, 2021, at 6:17 AM, Stefan Hajnoczi <stefa...@redhat.com> wrote: > > > > > > On Wed, Dec 15, 2021 at 10:35:31AM -0500, Jagannathan Raman wrote: > > >> @@ -114,6 +118,62 @@ static void vfu_object_set_device(Object *obj, > > const char *str, Error **errp) > > >> vfu_object_init_ctx(o, errp); > > >> } > > >> > > >> +static void vfu_object_ctx_run(void *opaque) > > >> +{ > > >> + VfuObject *o = opaque; > > >> + int ret = -1; > > >> + > > >> + while (ret != 0) { > > >> + ret = vfu_run_ctx(o->vfu_ctx); > > >> + if (ret < 0) { > > >> + if (errno == EINTR) { > > >> + continue; > > >> + } else if (errno == ENOTCONN) { > > >> + qemu_set_fd_handler(o->vfu_poll_fd, NULL, NULL, NULL); > > >> + o->vfu_poll_fd = -1; > > >> + object_unparent(OBJECT(o)); > > >> + break; > > > > > > If nothing else logs a message then I think that should be done here so > > > users know why their vfio-user server object disappeared. > > > > Sure will do. > > > > Do you prefer a trace, or a message to the console? Trace makes sense to me. > > Presently, the client could unplug the vfio-user device which would trigger > > the > > deletion of this object. This process could happen quietly. > > > > > > > >> + } else { > > >> + error_setg(&error_abort, "vfu: Failed to run device %s > > >> - %s", > > >> + o->device, strerror(errno)); > > > > > > error_abort is equivalent to assuming !o->daemon. In the case where the > > > user doesn't want to automatically shut down the process we need to log > > > a message without aborting. > > > > OK, makes sense. > > > > > > > >> + break; > > > > > > Indentation is off. > > > > > >> + } > > >> + } > > >> + } > > >> +} > > >> + > > >> +static void vfu_object_attach_ctx(void *opaque) > > >> +{ > > >> + VfuObject *o = opaque; > > >> + GPollFD pfds[1]; > > >> + int ret; > > >> + > > >> + qemu_set_fd_handler(o->vfu_poll_fd, NULL, NULL, NULL); > > >> + > > >> + pfds[0].fd = o->vfu_poll_fd; > > >> + pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR; > > >> + > > >> +retry_attach: > > >> + ret = vfu_attach_ctx(o->vfu_ctx); > > >> + if (ret < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) { > > >> + qemu_poll_ns(pfds, 1, 500 * (int64_t)SCALE_MS); > > >> + goto retry_attach; > > > > > > This can block the thread indefinitely. Other events like monitor > > > commands are not handled in this loop. Please make this asynchronous > > > (set an fd handler and return from this function so we can try again > > > later). > > > > > > The vfu_attach_ctx() implementation synchronously negotiates the > > > vfio-user connection :(. That's a shame because even if accept(2) is > > > handled asynchronously, the negotiation can still block. It would be > > > cleanest to have a fully async libvfio-user's vfu_attach_ctx() API to > > > avoid blocking. Is that possible? > > > > Thanos / John, > > > > Any thoughts on this? > > I'm discussing this with John and FYI there are other places where > libvfio-user can block, e.g. sending a response or receiving a command. Is it > just the negotiation you want it to be asynchronous or _all_ libvfio-user > operations? Making libvfio-user fully asynchronous might require a > substantial API rewrite.
I see at least two reasons for a fully async API: 1. The program wants to handle other events (e.g. a management REST API) from the same event loop thread that invokes libvfio-user. If libvfio-user blocks then the other events cannot be handled within a reasonable time frame. The workaround for this is to use multi-threading and ignore the event-driven architecture implied by vfu_get_poll_fd(). 2. The program handles multiple clients that do not trust each other. This could be a software-defined network switch or storage appliance. A malicious client can cause a denial-of-service by making a libvfio-user call block. Again, the program needs separate threads instead of an event loop to work around this. The downside to a sync approach is that programs that already have an event loop require extra code to set up dedicated threads for libvfio-user. That's a library integration/usability issue. In some cases it's okay to block: when the program doesn't need to handle other events. If most users of libvfio-user are expected to fall into this category then there's no need to change the API. Either way, the doc comments in libvfio-user.h aren't very clear. Someone integrating this library may think vfu_get_poll_fd() allows for fully async operation. Stefan
signature.asc
Description: PGP signature