On Thu, Aug 31, 2017 at 11:50:05AM +0200, Maxime Coquelin wrote:
> virtio_net device might be accessed while being reallocated
> in case of NUMA awareness. This case might be theoretical,
> but it will be needed anyway to protect vrings pages against
> invalidation.
>
> The virtio_net devs are now protected with a readers/writers
> lock, so that before reallocating the device, it is ensured
> that it is not being referenced by the processing threads.
>
[...]
>
> +struct virtio_net *
> +get_device(int vid)
> +{
> + struct virtio_net *dev;
> +
> + rte_rwlock_read_lock(&vhost_devices[vid].lock);
> +
> + dev = __get_device(vid);
> + if (unlikely(!dev))
> + rte_rwlock_read_unlock(&vhost_devices[vid].lock);
> +
> + return dev;
> +}
> +
> +void
> +put_device(int vid)
> +{
> + rte_rwlock_read_unlock(&vhost_devices[vid].lock);
> +}
> +
This patch introduced a per-device rwlock which needs to be acquired
unconditionally in the data path. So for each vhost device, the IO
threads of different queues will need to acquire/release this lock
during each enqueue and dequeue operation, which will cause cache
contention when multiple queues are enabled and handled by different
cores. With this patch alone, I saw ~7% performance drop when enabling
6 queues to do 64bytes iofwd loopback test. Is there any way to avoid
introducing this lock to the data path?
Best regards,
Tiwei Bie
