On Wed, Sep 27, 2023 at 9:51 PM Ilya Maximets <i.maxim...@ovn.org> wrote:
>
> We do not need the most up to date number of heads, we only want to
> know if there is at least one.
>
> Use shadow variable as long as it is not equal to the last available
> index checked. This avoids expensive qatomic dereference of the
> RCU-protected memory region cache as well as the memory access itself.
>
> The change improves performance of the af-xdp network backend by 2-3%.
>
> Signed-off-by: Ilya Maximets <i.maxim...@ovn.org>
Acked-by: Jason Wang <jasow...@redhat.com>
Thanks
> ---
>
> Version 2:
> - Changed to not skip error checks and a barrier.
> - Added comments about the need for a barrier.
>
> hw/virtio/virtio.c | 18 +++++++++++++++---
> 1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 4577f3f5b3..8a4c3e95d2 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -999,7 +999,12 @@ void virtqueue_push(VirtQueue *vq, const
> VirtQueueElement *elem,
> /* Called within rcu_read_lock(). */
> static int virtqueue_num_heads(VirtQueue *vq, unsigned int idx)
> {
> - uint16_t num_heads = vring_avail_idx(vq) - idx;
> + uint16_t avail_idx, num_heads;
> +
> + /* Use shadow index whenever possible. */
> + avail_idx = (vq->shadow_avail_idx != idx) ? vq->shadow_avail_idx
> + : vring_avail_idx(vq);
> + num_heads = avail_idx - idx;
>
> /* Check it isn't doing very strange things with descriptor numbers. */
> if (num_heads > vq->vring.num) {
> @@ -1007,8 +1012,15 @@ static int virtqueue_num_heads(VirtQueue *vq, unsigned
> int idx)
> idx, vq->shadow_avail_idx);
> return -EINVAL;
> }
> - /* On success, callers read a descriptor at vq->last_avail_idx.
> - * Make sure descriptor read does not bypass avail index read. */
> + /*
> + * On success, callers read a descriptor at vq->last_avail_idx.
> + * Make sure descriptor read does not bypass avail index read.
> + *
> + * This is necessary even if we are using a shadow index, since
> + * the shadow index could have been initialized by calling
> + * vring_avail_idx() outside of this function, i.e., by a guest
> + * memory read not accompanied by a barrier.
> + */
> if (num_heads) {
> smp_rmb();
> }
> --
> 2.41.0
>
