[PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent IOTLB mapping across device reset. There are two cases that backend may claim this feature bit on: - parent device that has to work with platform IOMMU - parent device with on-chip IOMMU that has

[PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1 mapping will be implicitly destroyed when the first .set_map call is invoked. Everytime when the .reset callback is invoked, any mapping left behind will be dropped then reset back to the initial 1:

[PATCH RFC v3 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
Devices with on-chip IOMMU may need to restore iotlb to 1:1 identity mapping from IOVA to PA. Before vhost-vdpa is going away, give them a chance to clean up and reset iotlb back to 1:1 identify mapping mode. This is done so that any vdpa bus driver may start with 1:1 identity mapping by default.

[PATCH RFC v3 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux/vdp

[PATCH RFC v3 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
In order to reduce needlessly high setup and teardown cost of iotlb mapping during live migration, it's crucial to decouple the vhost-vdpa iotlb abstraction from the virtio device life cycle, i.e. iotlb mappings should be left intact across virtio device reset [1]. For it to work, the on-chip IOMMU

[PATCH RFC v2 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent IOTLB mapping across device reset. There are two cases that backend may claim this feature bit on: - parent device that has to work with platform IOMMU - parent device with on-chip IOMMU that has

[PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux/vdp

[PATCH RFC v2 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
In order to reduce needlessly high setup and teardown cost of iotlb mapping during live migration, it's crucial to decouple the vhost-vdpa iotlb abstraction from the virtio device life cycle, i.e. iotlb mappings should be left intact across virtio device reset [1]. For it to work, the on-chip IOMMU

[PATCH RFC v2 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1 mapping will be implicitly destroyed when the first .set_map call is invoked. Everytime when the .reset callback is invoked, any mapping left behind will be dropped then reset back to the initial 1:

[PATCH RFC v2 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
Devices with on-chip IOMMU may need to restore iotlb to 1:1 identity mapping from IOVA to PA. Before vhost-vdpa is going away, give them a chance to clean up and reset iotlb back to 1:1 identify mapping mode. This is done so that any vdpa bus driver may start with 1:1 identity mapping by default.

[PATCH RFC v2 3/3] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-09-09 Thread Si-Wei Liu
With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descriptor table. Signed-o

[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group

2023-09-09 Thread Si-Wei Liu
Following patchset introduces dedicated group for descriptor table to reduce live migration downtime when passthrough VQ is being switched to shadow VQ. This RFC v2 is sent to incorporate the early feedback from reviewers on the uAPI and driver API part of changes, the associated driver patch set c

[PATCH RFC v2 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-09-09 Thread Si-Wei Liu
Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASID feature hasn't be

[PATCH RFC v2 1/3] vdpa: introduce dedicated descriptor group for virtqueue

2023-09-09 Thread Si-Wei Liu
In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already established termino