On Mon, 29 Aug 2022 08:56:58 +0800
Wenwu Ma <wenwux...@intel.com> wrote:
> Offloading small packets to DMA degrades throughput 10%~20%,
> and this is because DMA offloading is not free and DMA is not
> good at processing small packets. In addition, control plane
> packets are usually small, and assign those packets to DMA will
> significantly increase latency, which may cause timeout like
> TCP handshake packets. Therefore, this patch use CPU to perform
> small copies in vhost.
>
> Signed-off-by: Wenwu Ma <wenwux...@intel.com>
> ---
> v4:
> * fix coding style issues
> v3:
> * compare threshold with entire packet length
> v2:
> * fix CI build error
> ---
> lib/vhost/vhost.h | 7 ++--
> lib/vhost/virtio_net.c | 73 +++++++++++++++++++++++++++++++++---------
> 2 files changed, 62 insertions(+), 18 deletions(-)
>
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 40fac3b7c6..8a7d90f737 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -142,8 +142,10 @@ struct virtqueue_stats {
> * iovec
> */
> struct vhost_iovec {
> - void *src_addr;
> - void *dst_addr;
> + void *src_iov_addr;
> + void *dst_iov_addr;
> + void *src_virt_addr;
> + void *dst_virt_addr;
> size_t len;
> };
>
> @@ -155,6 +157,7 @@ struct vhost_iov_iter {
> struct vhost_iovec *iov;
> /** number of iovec in this iterator */
> unsigned long nr_segs;
> + unsigned long nr_len;
> };
>
> struct async_dma_vchan_info {
> diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c
> index 35fa4670fd..cf796183a0 100644
> --- a/lib/vhost/virtio_net.c
> +++ b/lib/vhost/virtio_net.c
> @@ -26,6 +26,8 @@
>
> #define MAX_BATCH_LEN 256
>
> +#define CPU_COPY_THRESHOLD_LEN 256
Good idea.
This heuristic matches what Linux network drivers do to avoid
creating large buffers from small packets.
Patch no longer applies cleanly to main branch (after 2 years).
Also the copy threshold should be configurable via dev_args to allow
for testing and CPU differences.
