Hi, I recently did some stress tests of a vhost-user interface using an UDP traffic generator. Traffic generator was connected to 2 physical ports that are in turn connected to 2 virtio ports through a linux bridge, VM (running linux) doing routing to forward packets between the 2 virtio ports. When traffic reaches high pps rates of small packets, I faced the 2 following problems:
- at some time, my qemu socket becomes full, causing qemu to send incomplete SET_VRING_CALL messages to vhost-user backend (without proper fd set in ancillary data). - after some time, some interrupts are lost, causing the VM to stop transmitting packets. Both problems come from the fact that interrupt masking/unmasking of the VM is deferred to vhost-user backend through the linux socket. First problem comes from the fact that socket buffer gets full; it is corrected in the first patch of the serie. Second problem is a bit more complex. From what i understand of the code, when VM wants to mask/unmask interrupts, qemu traps the command and sends a SET_VRING_CALL to the vhost-user to swap interrupt notifications either to a dummy descriptor or to an fd that was given to kvm module to route interruption to the VM. After sending SET_VRING_CALL message through the socket, VM code continues to run, assuming that the interrupts are now masked; but due to linux socket, this message may be buffered and not currently treated by the vhost-user backend, ie interrupts are not really masked/unmasked as they ought to be. I think it can be solved in two different ways: - by waiting for an acknowledgement of vhost-user backend before exiting interrupt masking function, this ensures that interrupt masking is correctly taken into account before giving back hand to the VM, but it has a very high cost in term of cycles. Moreover, unless specifying a new option, it will break current API, since existing vhost-user implementations do not expect a return message on a SET_VRING_CALL call. - second way could be, in case vhost-user is involved, to restore the initial behaviour of interrupt masking (ie masking/unmasking interrupts by taking/giving vring_call fd from/to kernel kvm module). Patches 2 and 3 of the serie propose an implementation of this second option. Didier Didier Pallard (3): char: fix vhost-user socket full virtio-pci: add an option to bypass guest_notifier_mask vhost-net: force guest_notifier_mask bypass in vhost-user case hw/net/vhost_net.c | 19 ++++++++++++++++++- hw/virtio/vhost.c | 13 +++++++++++++ hw/virtio/virtio-pci.c | 29 +++++++++++++++++++++++------ hw/virtio/virtio-pci.h | 6 ++++++ qemu-char.c | 10 ++++++++++ 5 files changed, 70 insertions(+), 7 deletions(-) -- 2.1.4
