> -----Original Message-----
> From: Maxime Coquelin <maxime.coque...@redhat.com>
> Sent: Friday, June 18, 2021 10:04 PM
> To: dev@dpdk.org; david.march...@redhat.com; Xia, Chenbo
> <chenbo....@intel.com>
> Cc: Maxime Coquelin <maxime.coque...@redhat.com>; sta...@dpdk.org
> Subject: [PATCH v6 1/7] vhost: fix missing memory table NUMA realloc
>
> When the guest allocates virtqueues on a different NUMA node
> than the one the Vhost metadata are allocated, both the Vhost
> device struct and the virtqueues struct are reallocated.
>
> However, reallocating the Vhost memory table was missing, which
> likely causes at least one cross-NUMA accesses for every burst
> of packets.
>
> This patch reallocates this table on the same NUMA node as the
> other metadata.
>
> Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
> Cc: sta...@dpdk.org
>
> Reported-by: David Marchand <david.march...@redhat.com>
> Signed-off-by: Maxime Coquelin <maxime.coque...@redhat.com>
> ---
> lib/vhost/vhost_user.c | 19 +++++++++++++++++--
> 1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> index 8f0eba6412..b5a84f3dcd 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -473,8 +473,8 @@ vhost_user_set_vring_num(struct virtio_net **pdev,
> }
>
> /*
> - * Reallocate virtio_dev and vhost_virtqueue data structure to make them on
> the
> - * same numa node as the memory of vring descriptor.
> + * Reallocate virtio_dev, vhost_virtqueue and related data structures to
> + * make them on the same numa node as the memory of vring descriptor.
> */
> #ifdef RTE_LIBRTE_VHOST_NUMA
> static struct virtio_net*
> @@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index)
> goto out;
> }
> if (oldnode != newnode) {
> + struct rte_vhost_memory *old_mem;
> + ssize_t mem_size;
> +
> VHOST_LOG_CONFIG(INFO,
> "reallocate dev from %d to %d node\n",
> oldnode, newnode);
> @@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index)
>
> memcpy(dev, old_dev, sizeof(*dev));
> rte_free(old_dev);
> +
> + mem_size = sizeof(struct rte_vhost_memory) +
> + sizeof(struct rte_vhost_mem_region) *
> dev->mem->nregions;
> + old_mem = dev->mem;
> + dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode);
> + if (!dev->mem) {
> + dev->mem = old_mem;
> + goto out;
> + }
> +
> + memcpy(dev->mem, old_mem, mem_size);
> + rte_free(old_mem);
> }
>
> out:
> --
> 2.31.1
Reviewed-by: Chenbo Xia <chenbo....@intel.com>
