Re: [PATCH] mlx5_vdpa: offer VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK

2023-07-03 Thread Si-Wei Liu
On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: Offer this backend feature as mlx5 is compatible with it. It allows it to do live migration with CVQ, dynamically switching between passthrough and shadow virtqueue. Signed-off-by: Eu

Re: [PATCH] vdpa: reject F_ENABLE_AFTER_DRIVER_OK if backend does not support it

2023-07-19 Thread Si-Wei Liu
On 7/5/2023 11:07 PM, Michael S. Tsirkin wrote: On Wed, Jul 05, 2023 at 05:07:11PM -0700, Shannon Nelson wrote: On 7/5/23 11:27 AM, Eugenio Perez Martin wrote: On Wed, Jul 5, 2023 at 9:50 AM Jason Wang wrote: On Tue, Jul 4, 2023 at 11:45 PM Michael S. Tsirkin wrote: On Tue, Jul 04, 2023 a

Re: [PATCH] vdpa: reject F_ENABLE_AFTER_DRIVER_OK if backend does not support it

2023-07-19 Thread Si-Wei Liu
On 7/19/2023 3:26 PM, Michael S. Tsirkin wrote: On Wed, Jul 19, 2023 at 03:20:03PM -0700, Si-Wei Liu wrote: On 7/5/2023 11:07 PM, Michael S. Tsirkin wrote: On Wed, Jul 05, 2023 at 05:07:11PM -0700, Shannon Nelson wrote: On 7/5/23 11:27 AM, Eugenio Perez Martin wrote: On Wed, Jul 5, 2023

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-03 Thread Si-Wei Liu
On 8/3/2023 1:03 AM, Jason Wang wrote: On Thu, Aug 3, 2023 at 1:13 AM Dragos Tatulea wrote: The mr->initialized flag is shared between the control vq and data vq part of the mr init/uninit. But if the control vq and data vq get placed in different ASIDs, it can happen that initializing the co

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-08 Thread Si-Wei Liu
On 8/7/2023 8:00 PM, Jason Wang wrote: On Fri, Aug 4, 2023 at 1:58 AM Si-Wei Liu wrote: On 8/3/2023 1:03 AM, Jason Wang wrote: On Thu, Aug 3, 2023 at 1:13 AM Dragos Tatulea wrote: The mr->initialized flag is shared between the control vq and data vq part of the mr init/uninit. But

[PATCH RFC 3/3] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-08-09 Thread Si-Wei Liu
-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 10 ++ include/uapi/linux/vhost.h | 8 2 files changed, 18 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index f2e5dce..eabac06 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -602,6

[PATCH RFC 1/3] vdpa: introduce dedicated descriptor group for virtqueue

2023-08-09 Thread Si-Wei Liu
ing descriptors. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index db1b0ea..17a4efa 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -204,6 +204,16 @@ struc

[PATCH RFC 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-08-09 Thread Si-Wei Liu
asn't been exposed will result in an error. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + include/uapi/linux/vhost_types.h | 5 + 2 files changed, 22 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index b43e868..f2e5dce 10

[PATCH RFC 0/3] vdpa: dedicated descriptor table group

2023-08-09 Thread Si-Wei Liu
would always belong to a same group (1). | data vq | ctrl vq ==+==+=== vq_group |0 |1 vq_desc_group |2 |1 --- Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-09 Thread Si-Wei Liu
On 8/8/2023 11:52 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 6:58 AM Si-Wei Liu wrote: On 8/7/2023 8:00 PM, Jason Wang wrote: On Fri, Aug 4, 2023 at 1:58 AM Si-Wei Liu wrote: On 8/3/2023 1:03 AM, Jason Wang wrote: On Thu, Aug 3, 2023 at 1:13 AM Dragos Tatulea wrote: The mr

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-10 Thread Si-Wei Liu
On 8/9/2023 8:10 PM, Jason Wang wrote: On Thu, Aug 10, 2023 at 8:40 AM Si-Wei Liu wrote: On 8/8/2023 11:52 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 6:58 AM Si-Wei Liu wrote: On 8/7/2023 8:00 PM, Jason Wang wrote: On Fri, Aug 4, 2023 at 1:58 AM Si-Wei Liu wrote: On 8/3/2023 1:03

[PATCH RFC 2/4] vdpa/mlx5: implement .reset_map driver op

2023-08-14 Thread Si-Wei Liu
This patch is based on top of the "vdpa/mlx5: Fixes for ASID handling" series [1]. [1] vdpa/mlx5: Fixes for ASID handling https://lore.kernel.org/virtualization/20230802171231.11001-1-dtatu...@nvidia.com/ Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + dr

[PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-14 Thread Si-Wei Liu
Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 +++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 62b0a01..75092a7 100644 --- a/drivers/vhost/vdpa.c

[PATCH RFC 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-08-14 Thread Si-Wei Liu
pping when the first .set_map or .dma_map callback is invoked. [1] Reducing vdpa migration downtime because of memory pin / maps https://www.mail-archive.com/qemu-devel@nongnu.org/msg953755.html --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vdpa/mlx5: implement .rese

[PATCH RFC 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-08-14 Thread Si-Wei Liu
Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index b43e868..62b0a01 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -131,6 +131,15 @@ static struct

[PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-14 Thread Si-Wei Liu
Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index db1b0ea..3a3878d 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -314,6 +314,12 @@ struct vdpa_map_file

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index db1b0ea..3a3878d 100644 --- a/include

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 +++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a

Re: [PATCH RFC 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:32 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index b43e868..62b0a01 100644

Re: [PATCH RFC 2/4] vdpa/mlx5: implement .reset_map driver op

2023-08-15 Thread Si-Wei Liu
On 8/15/2023 1:26 AM, Dragos Tatulea wrote: On Mon, 2023-08-14 at 18:43 -0700, Si-Wei Liu wrote: This patch is based on top of the "vdpa/mlx5: Fixes for ASID handling" series [1]. [1] vdpa/mlx5: Fixes for ASID handling https://lore.kernel.org/virtualization/20230802171231.110

Re: [PATCH RFC 0/3] vdpa: dedicated descriptor table group

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 7:49 AM, Eugenio Perez Martin wrote: On Wed, Aug 9, 2023 at 2:56 PM Si-Wei Liu wrote: Following patchset introduces dedicated group for descriptor table to reduce live migration downtime when passthrough VQ is being switched to shadow VQ. As this RFC set is to seek early feedback

Re: [PATCH RFC 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 8:49 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 8:56 PM Si-Wei Liu wrote: Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation cal

Re: [PATCH RFC 0/3] vdpa: dedicated descriptor table group

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 8:50 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 8:56 PM Si-Wei Liu wrote: Following patchset introduces dedicated group for descriptor table to reduce live migration downtime when passthrough VQ is being switched to shadow VQ. As this RFC set is to seek early feedback on the

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-16 Thread Si-Wei Liu
On 8/15/2023 6:48 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 6:31 AM Si-Wei Liu wrote: On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 +++- include/uapi

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-16 Thread Si-Wei Liu
On 8/15/2023 6:55 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 3:49 AM Si-Wei Liu wrote: On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-21 Thread Si-Wei Liu
On 8/17/2023 8:28 AM, Eugenio Perez Martin wrote: On Thu, Aug 17, 2023 at 2:05 AM Si-Wei Liu wrote: On 8/15/2023 6:55 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 3:49 AM Si-Wei Liu wrote: On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-28 Thread Si-Wei Liu
On 8/22/2023 1:54 AM, Jason Wang wrote: On Thu, Aug 17, 2023 at 7:44 AM Si-Wei Liu wrote: On 8/15/2023 6:48 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 6:31 AM Si-Wei Liu wrote: On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by

Re: [PATCH] vdpa: consume device_features parameter

2023-09-07 Thread Si-Wei Liu
xdeadbeef" > > Fixes: a4442ce58ebb ("vdpa: allow provisioning device features") > Signed-off-by: Allen Hubbe > Reviewed-by: Shannon Nelson Reviewed-by: Si-Wei Liu > --- >  vdpa/vdpa.c | 2 ++ >  1 file changed, 2 insertions(+) > > diff --git a/vdpa/vdpa.c

Re: [PATCH] vdpa: consume device_features parameter

2023-09-08 Thread Si-Wei Liu
On 9/7/2023 5:07 PM, David Ahern wrote: On 9/7/23 2:41 PM, Si-Wei Liu wrote: Hi David, Why this patch doesn't get picked in the last 4 months? Maybe the subject is not clear, but this is an iproute2 patch. Would it be possible to merge at your earliest convenience? PS, adding my R-b t

[PATCH RFC v2 1/3] vdpa: introduce dedicated descriptor group for virtqueue

2023-09-09 Thread Si-Wei Liu
to the default address space, i.e. in ASID 0. QEMU's shadow virtqueue is going to utilize dedicated descriptor group to speed up map and unmap operations, yielding tremendous downtime reduction by avoiding the full and slow remap cycle in SVQ switching. Signed-off-by: Si-Wei Liu Acked-by: Eugeni

[PATCH RFC v2 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-09-09 Thread Si-Wei Liu
asn't been exposed will result in an error. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- RFC v1 -> v2: - add clarifications for what areas F_DESC_ASID should cover --- drivers/vhost/vdpa.c | 17 + include/uapi/linux/vhost_types.h | 5 + 2 file

[PATCH RFC v2 3/3] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-09-09 Thread Si-Wei Liu
-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c | 10 ++ include/uapi/linux/vhost.h | 8 2 files changed, 18 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index f2e5dce..eabac06 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers

[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group

2023-09-09 Thread Si-Wei Liu
|1 vq_desc_group |2 |1 --- Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor group backend feature vhost-vdpa: uAPI to get dedicated descriptor group id drivers/vhost/vdpa.c | 27

[PATCH RFC v2 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index eabac06..71fbd559 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -131,6 +131,15 @@ static struct

[PATCH RFC v2 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
mapping, then the device .reset routine can run free from having to clean up memory mappings. Signed-off-by: Si-Wei Liu --- RFC v1 -> v2: - fix error path when both CVQ and DVQ fall in same asid --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c|

[PATCH RFC v2 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
up" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vdpa/mlx5: implement .reset_map driver op vhost-vdpa: should restore 1:1 dma mapping before detaching driv

[PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux

[PATCH RFC v2 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
has the expected .reset_map support in driver Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 15 ++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 71fbd559

[PATCH RFC v3 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vdpa/mlx5: implement .reset_map driver op vhost-vdpa: s

[PATCH RFC v3 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux

[PATCH RFC v3 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index eabac06..71fbd559 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -131,6 +131,15 @@ static struct

[PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
has the expected .reset_map support in driver Signed-off-by: Si-Wei Liu --- RFC v2 -> v3: - fix missing return due to merge error --- drivers/vhost/vdpa.c | 16 +++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --gi

[PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
mapping, then the device .reset routine can run free from having to clean up memory mappings. Signed-off-by: Si-Wei Liu --- RFC v1 -> v2: - fix error path when both CVQ and DVQ fall in same asid --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c|

Re: [PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-11 Thread Si-Wei Liu
Hi Jason, On 9/10/2023 8:42 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Sep 9, 2023 at 9:34 PM Si-Wei Liu wrote: On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. As discussed before. On-chip IOMMU is the hardware details that need to be hidden by the

Re: [PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-11 Thread Si-Wei Liu
On 9/10/2023 8:48 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1 mapping will be implicitly destroyed when the first .set_map call is invoked. Everytime when the

Re: [PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-11 Thread Si-Wei Liu
On 9/10/2023 8:52 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent IOTLB mapping across device reset. As discussed, the IOTLB persists for devices with platform IOMMU

Re: [PATCH] vdpa: consume device_features parameter

2023-09-11 Thread Si-Wei Liu
Thanks David, for clarifications. Now I see the patch just got posted by Shannon (thanks!) with the correct iproute2 label in the subject line. We may expect to see this land on iproute2 repo soon? Thanks! -Siwei On 9/9/2023 1:36 PM, David Ahern wrote: On 9/8/23 12:37 PM, Si-Wei Liu wrote

Re: [PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-14 Thread Si-Wei Liu
On 9/11/2023 11:23 PM, Jason Wang wrote: On Tue, Sep 12, 2023 at 7:31 AM Si-Wei Liu wrote: Hi Jason, On 9/10/2023 8:42 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Sep 9, 2023 at 9:34 PM Si-Wei Liu wrote: On-chip IOMMU parent driver could use it to restore memory mapping to the initial

Re: [PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-14 Thread Si-Wei Liu
On 9/11/2023 11:53 PM, Jason Wang wrote: On Tue, Sep 12, 2023 at 8:02 AM Si-Wei Liu wrote: On 9/10/2023 8:48 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1

Re: [PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-14 Thread Si-Wei Liu
On 9/12/2023 12:01 AM, Jason Wang wrote: On Tue, Sep 12, 2023 at 8:28 AM Si-Wei Liu wrote: On 9/10/2023 8:52 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-14 Thread Si-Wei Liu
pdate flow vdpa/mlx5: Introduce mr for vq descriptor vdpa/mlx5: Enable hw support for vq descriptor mapping vdpa/mlx5: Make iotlb helper functions more generic vdpa/mlx5: Update cvq iotlb mapping on ASID change Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-26 Thread Si-Wei Liu
mr for vq descriptor   vdpa/mlx5: Enable hw support for vq descriptor mapping   vdpa/mlx5: Make iotlb helper functions more generic   vdpa/mlx5: Update cvq iotlb mapping on ASID change   Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-Wei Liu (3):   vdpa: introduce dedicated

[PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-10 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 1 file ch

[PATCH 1/4] vdpa: introduce .reset_map operation callback

2023-10-10 Thread Si-Wei Liu
: Si-Wei Liu --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index d376309..26ae6ae 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -327,6 +327,15 @@ struct vdpa_map_file

[PATCH 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-10 Thread Si-Wei Liu
eries: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vhost-vdpa: reset vendor specific mapping to initial state in .release vhost-vdpa: introduce IOTLB_PERSIST backend feature

[PATCH 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-10 Thread Si-Wei Liu
es on-chip IOMMU to perform 1:1 passthrough translation from PA to IOVA as-is to begin with, and .reset_map is the only means to turn back iotlb to the identity mapping mode after vhost-vdpa is gone. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 15 +++ include/

[PATCH 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-10 Thread Si-Wei Liu
by itself. The cvq mapping also needs to be cleared if is in the given ASID. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 17 + drivers/vdp

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-11 Thread Si-Wei Liu
On 10/11/2023 4:21 AM, Eugenio Perez Martin wrote: On Tue, Oct 10, 2023 at 11:05 AM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to the initial or default state using the .reset_map op, as it's desirable for

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to the initial or default state using the .reset_map op, as it's desirable for some parent de

Re: [PATCH 1/4] vdpa: introduce .reset_map operation callback

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 7:49 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Device specific IOMMU parent driver who wishes to see mapping to be decoupled from virtio or vdpa device life cycle (device reset) can use it to restore memory mapping in the device IOMMU to the

Re: [PATCH 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 8:04 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Since commit 6f5312f80183 ("vdpa/mlx5: Add support for running with virtio_vdpa"), mlx5_vdpa starts with preallocate 1:1 DMA MR at device creation time. This 1:1 DMA MR will be implicitly

[RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-13 Thread Si-Wei Liu
RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei@oracle.com/ Signed-off-by: Si-Wei Liu

Re: [RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-13 Thread Si-Wei Liu
Hi Stefano, On 10/13/2023 2:22 AM, Stefano Garzarella wrote: Hi Si-Wei, On Fri, Oct 13, 2023 at 01:23:40AM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. I can test it, but what I should stress? Great, thank you! As you see, my patch moved

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
On 10/15/2023 11:32 PM, Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor

[PATCH v2 1/4] vdpa: introduce .reset_map operation callback

2023-10-16 Thread Si-Wei Liu
have also implemented their own DMA ops, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping, either. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+

[PATCH v2 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c

[PATCH v2 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-16 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v2 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-16 Thread Si-Wei Liu
eturn due to merge error in patch #4 RFC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map ope

[PATCH v2 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-16 Thread Si-Wei Liu
by itself. The cvq mapping also needs to be cleared if is in the given ASID. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 17

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-17 Thread Si-Wei Liu
On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote: On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote

[RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-17 Thread Si-Wei Liu
RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei@oracle.com/ Signed-off-by: Si-Wei Liu

Re: [RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-17 Thread Si-Wei Liu
Hi Stefano, On 10/17/2023 6:44 AM, Stefano Garzarella wrote: On Fri, Oct 13, 2023 at 10:29:26AM -0700, Si-Wei Liu wrote: Hi Stefano, On 10/13/2023 2:22 AM, Stefano Garzarella wrote: Hi Si-Wei, On Fri, Oct 13, 2023 at 01:23:40AM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/17/2023 10:27 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 12:36 PM Si-Wei Liu wrote: On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote: On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 12:00 AM, Jason Wang wrote: Unfortunately, it's a must to stick to ABI. I agree it's a mess but we don't have a better choice. Or we can fail the probe if userspace doesn't ack this feature. Antoher idea we can just do the following in vhost_vdpa reset? config->reset() if (IOTL

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 4:14 AM, Eugenio Perez Martin wrote: On Wed, Oct 18, 2023 at 10:44 AM Si-Wei Liu wrote: On 10/17/2023 10:27 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 12:36 PM Si-Wei Liu wrote: On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote

Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 1:05 AM, Stefano Garzarella wrote: On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https

[PATCH v3 4/5] vdpa/mlx5: implement .reset_map driver op

2023-10-18 Thread Si-Wei Liu
d clean up memory mappings by itself. Add a module paramemter, persist_mapping, to cater for older userspace which may wish to see mapping to be cleared during reset. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1

[PATCH v3 2/5] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c

[PATCH v3 3/5] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-18 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v3 1/5] vdpa: introduce .reset_map operation callback

2023-10-18 Thread Si-Wei Liu
have also implemented their own DMA ops, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping, either. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+

[PATCH v3 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-18 Thread Si-Wei Liu
FC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (5): vdpa: introduce .reset_map operation callback vhost-vdpa: reset

[PATCH v3 5/5] vdpa_sim: implement .reset_map support

2023-10-18 Thread Si-Wei Liu
Add a module paramemter, iotlb_persist, to cater for older userspace which may wish to see mapping to be cleared during reset. Signed-off-by: Si-Wei Liu Tested-by: Stefano Garzarella --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 58 ++-- 1 file changed, 47 insertions(+

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 7:53 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 4:49 PM Si-Wei Liu wrote: On 10/18/2023 12:00 AM, Jason Wang wrote: Unfortunately, it's a must to stick to ABI. I agree it's a mess but we don't have a better choice. Or we can fail the probe if userspace d

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 7:39 AM, Eugenio Perez Martin wrote: On Thu, Oct 19, 2023 at 10:27 AM Jason Wang wrote: On Thu, Oct 19, 2023 at 2:47 PM Si-Wei Liu wrote: On 10/18/2023 7:53 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 4:49 PM Si-Wei Liu wrote: On 10/18/2023 12:00 AM, Jason Wang wrote

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/17/2023 10:27 PM, Jason Wang wrote: If we do this without a negotiation, IOTLB will not be clear but the Qemu will try to re-program the IOTLB after reset. Which will break? 1) stick the exact old behaviour with just one line of check It's not just one line of check here, the old be

Re: [PATCH vhost v4 00/16] vdpa: Add support for vq descriptor mappings

2023-10-19 Thread Si-Wei Liu
For patches 05-16: Reviewed-by: Si-Wei Liu Tested-by: Si-Wei Liu Thanks for the fixes! On 10/18/2023 10:14 AM, Dragos Tatulea wrote: This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using

Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 2:29 AM, Stefano Garzarella wrote: On Wed, Oct 18, 2023 at 04:47:48PM -0700, Si-Wei Liu wrote: On 10/18/2023 1:05 AM, Stefano Garzarella wrote: On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. Works

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 9:11 PM, Jason Wang wrote: On Fri, Oct 20, 2023 at 6:28 AM Si-Wei Liu wrote: On 10/19/2023 7:39 AM, Eugenio Perez Martin wrote: On Thu, Oct 19, 2023 at 10:27 AM Jason Wang wrote: On Thu, Oct 19, 2023 at 2:47 PM Si-Wei Liu wrote: On 10/18/2023 7:53 PM, Jason Wang wrote

[PATCH v4 4/7] vdpa: introduce .compat_reset operation callback

2023-10-21 Thread Si-Wei Liu
ot bother to implement such op, but only those drivers that are doing or used to do non-compliant map-cleaning reset will have to. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdp

[PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-21 Thread Si-Wei Liu
RFC v3: - fix missing return due to merge error in patch #4 RFC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (7):

[PATCH v4 1/7] vdpa: introduce .reset_map operation callback

2023-10-21 Thread Si-Wei Liu
ot a MUST for every parent that implements the .dma_map or .set_map API, because device may work with DMA ops directly by implement their own to manipulate system memory mappings, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping. Signed-off-

[PATCH v4 2/7] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-21 Thread Si-Wei Liu
evice reset should not cause mapping to go away on such IOTLB model, so persistent mapping is implied across reset. Before the userspace process using vhost-vdpa is gone, give it a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio

[PATCH v4 3/7] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-21 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v4 6/7] vdpa/mlx5: implement .reset_map driver op

2023-10-21 Thread Si-Wei Liu
d clean up memory mappings by itself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1

[PATCH v4 7/7] vdpa_sim: implement .reset_map support

2023-10-21 Thread Si-Wei Liu
tself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Signed-off-by: Si-Wei Liu Tested-by: Stefano Garzarella --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 52 ++-- 1 file changed, 43 insertions(

[PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-21 Thread Si-Wei Liu
ompared to overloading the existing .reset with flags, .compat_reset won't cause any extra burden to the implementation of every compliant driver. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h

Re: [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-23 Thread Si-Wei Liu
On 10/22/2023 8:51 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Oct 21, 2023 at 5:28 PM Si-Wei Liu wrote: In order to reduce needlessly high setup and teardown cost of iotlb mapping during live migration, it's crucial to decouple the vhost-vdpa iotlb abstraction from the virtio device

Re: [PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-23 Thread Si-Wei Liu
59 AM, Dragos Tatulea wrote: On Sat, 2023-10-21 at 02:25 -0700, Si-Wei Liu wrote: Using .compat_reset op from the previous patch, the buggy .reset behaviour can be kept as-is on older userspace apps, which don't ack the IOTLB_PERSIST backend feature. As this compatibility quirk is limite

[PATCH] vhost-vdpa: fix NULL pointer deref in _compat_vdpa_reset

2023-10-23 Thread Si-Wei Liu
om/ Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 9ce40003793b..9a2343c45df0 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -232,9 +232,11 @@ static i

  1   2   3   4   5   >