Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-16 Thread Shenming Lu
On 2021/7/16 9:20, Tian, Kevin wrote: > To summarize, for vIOMMU we can work with the spec owner to > define a proper interface to feedback such restriction into the guest > if necessary. For the kernel part, it's clear that IOMMU fd should > disallow two devices attached to a single [RID] or [R

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Shenming Lu
On 2021/7/15 14:49, Tian, Kevin wrote: >> From: Shenming Lu >> Sent: Thursday, July 15, 2021 2:29 PM >> >> On 2021/7/15 11:55, Tian, Kevin wrote: >>>> From: Shenming Lu >>>> Sent: Thursday, July 15, 2021 11:21 AM >>>> >>

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-14 Thread Shenming Lu
On 2021/7/15 11:55, Tian, Kevin wrote: >> From: Shenming Lu >> Sent: Thursday, July 15, 2021 11:21 AM >> >> On 2021/7/9 15:48, Tian, Kevin wrote: >>> 4.6. I/O page fault >>> +++ >>> >>> uAPI is TBD. Here is just about t

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-14 Thread Shenming Lu
On 2021/7/9 15:48, Tian, Kevin wrote: > 4.6. I/O page fault > +++ > > uAPI is TBD. Here is just about the high-level flow from host IOMMU driver > to guest IOMMU driver and backwards. This flow assumes that I/O page faults > are reported via IOMMU interrupts. Some devices report fa

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-07 Thread Shenming Lu
On 2021/6/7 20:19, Liu, Yi L wrote: >> From: Shenming Lu >> Sent: Friday, June 4, 2021 10:03 AM >> >> On 2021/6/4 2:19, Jacob Pan wrote: >>> Hi Shenming, >>> >>> On Wed, 2 Jun 2021 12:50:26 +0800, Shenming Lu >> >>> wrote: >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread Shenming Lu
On 2021/6/4 2:19, Jacob Pan wrote: > Hi Shenming, > > On Wed, 2 Jun 2021 12:50:26 +0800, Shenming Lu > wrote: > >> On 2021/6/2 1:33, Jason Gunthorpe wrote: >>> On Tue, Jun 01, 2021 at 08:30:35PM +0800, Lu Baolu wrote: >>> >>>> The drivers re

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-01 Thread Shenming Lu
On 2021/6/2 1:33, Jason Gunthorpe wrote: > On Tue, Jun 01, 2021 at 08:30:35PM +0800, Lu Baolu wrote: > >> The drivers register per page table fault handlers to /dev/ioasid which >> will then register itself to iommu core to listen and route the per- >> device I/O page faults. > > I'm still confu

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-01 Thread Shenming Lu
On 2021/6/1 20:30, Lu Baolu wrote: > On 2021/6/1 15:15, Shenming Lu wrote: >> On 2021/6/1 13:10, Lu Baolu wrote: >>> Hi Shenming, >>> >>> On 6/1/21 12:31 PM, Shenming Lu wrote: >>>> On 2021/5/27 15:58, Tian, Kevin wrote: >>>>> /dev/io

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-01 Thread Shenming Lu
On 2021/6/1 13:10, Lu Baolu wrote: > Hi Shenming, > > On 6/1/21 12:31 PM, Shenming Lu wrote: >> On 2021/5/27 15:58, Tian, Kevin wrote: >>> /dev/ioasid provides an unified interface for managing I/O page tables for >>> devices assigned to userspace. Device p

Re: [RFC PATCH v3 8/8] vfio: Add nested IOPF support

2021-05-31 Thread Shenming Lu
On 2021/5/27 19:18, Lu Baolu wrote: > Hi Shenming and Alex, > > On 5/27/21 7:03 PM, Shenming Lu wrote: >>> I haven't fully read all the references, but who imposes the fact that >>> there's only one fault handler per device?  If type1 could register one &g

Re: [RFC] /dev/ioasid uAPI proposal

2021-05-31 Thread Shenming Lu
On 2021/5/27 15:58, Tian, Kevin wrote: > /dev/ioasid provides an unified interface for managing I/O page tables for > devices assigned to userspace. Device passthrough frameworks (VFIO, vDPA, > etc.) are expected to use this interface instead of creating their own logic > to > isolate untrusted

Re: [RFC] /dev/ioasid uAPI proposal

2021-05-31 Thread Shenming Lu
On 2021/6/1 10:36, Jason Wang wrote: > > 在 2021/5/31 下午4:41, Liu Yi L 写道: >>> I guess VFIO_ATTACH_IOASID will fail if the underlayer doesn't support >>> hardware nesting. Or is there way to detect the capability before? >> I think it could fail in the IOASID_CREATE_NESTING. If the gpa_ioasid >> is

Re: [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough

2021-05-27 Thread Shenming Lu
On 2021/5/25 6:11, Alex Williamson wrote: > On Fri, 21 May 2021 14:37:21 +0800 > Shenming Lu wrote: > >> Hi Alex, >> >> On 2021/5/19 2:57, Alex Williamson wrote: >>> On Fri, 9 Apr 2021 11:44:12 +0800 >>> Shenming Lu wrote: >>> >>&

Re: [RFC PATCH v3 2/8] vfio/type1: Add a page fault handler

2021-05-27 Thread Shenming Lu
On 2021/5/25 6:11, Alex Williamson wrote: > On Fri, 21 May 2021 14:38:52 +0800 > Shenming Lu wrote: > >> On 2021/5/19 2:58, Alex Williamson wrote: >>> On Fri, 9 Apr 2021 11:44:14 +0800 >>> Shenming Lu wrote: >>> >>>> VFIO manages the

Re: [RFC PATCH v3 5/8] vfio/type1: VFIO_IOMMU_ENABLE_IOPF

2021-05-27 Thread Shenming Lu
On 2021/5/25 6:11, Alex Williamson wrote: > On Fri, 21 May 2021 14:38:25 +0800 > Shenming Lu wrote: > >> On 2021/5/19 2:58, Alex Williamson wrote: >>> On Fri, 9 Apr 2021 11:44:17 +0800 >>> Shenming Lu wrote: >>> >>>> Since enabling IOPF

Re: [RFC PATCH v3 8/8] vfio: Add nested IOPF support

2021-05-27 Thread Shenming Lu
On 2021/5/25 6:11, Alex Williamson wrote: > On Mon, 24 May 2021 21:11:11 +0800 > Shenming Lu wrote: > >> On 2021/5/21 15:59, Shenming Lu wrote: >>> On 2021/5/19 2:58, Alex Williamson wrote: >>>> On Fri, 9 Apr 2021 11:44:20 +0800 >>>> Shenmi

Re: [RFC PATCH v3 8/8] vfio: Add nested IOPF support

2021-05-24 Thread Shenming Lu
On 2021/5/21 15:59, Shenming Lu wrote: > On 2021/5/19 2:58, Alex Williamson wrote: >> On Fri, 9 Apr 2021 11:44:20 +0800 >> Shenming Lu wrote: >> >>> To set up nested mode, drivers such as vfio_pci need to register a >>> handler to receive stage/l

Re: [RFC PATCH v3 8/8] vfio: Add nested IOPF support

2021-05-21 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:20 +0800 > Shenming Lu wrote: > >> To set up nested mode, drivers such as vfio_pci need to register a >> handler to receive stage/level 1 faults from the IOMMU, but since >> currently each device

Re: [RFC PATCH v3 7/8] vfio/type1: Add selective DMA faulting support

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:19 +0800 > Shenming Lu wrote: > >> Some devices only allow selective DMA faulting. Similar to the selective >> dirty page tracking, the vendor driver can call vfio_pin_pages() to >> indicate the no

Re: [RFC PATCH v3 6/8] vfio/type1: No need to statically pin and map if IOPF enabled

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:18 +0800 > Shenming Lu wrote: > >> If IOPF enabled for the VFIO container, there is no need to statically >> pin and map the entire DMA range, we can do it on demand. And unmap >> according to

Re: [RFC PATCH v3 2/8] vfio/type1: Add a page fault handler

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:14 +0800 > Shenming Lu wrote: > >> VFIO manages the DMA mapping itself. To support IOPF (on-demand paging) >> for VFIO (IOMMU capable) devices, we add a VFIO page fault handler to >> serve the re

Re: [RFC PATCH v3 5/8] vfio/type1: VFIO_IOMMU_ENABLE_IOPF

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:17 +0800 > Shenming Lu wrote: > >> Since enabling IOPF for devices may lead to a slow ramp up of performance, >> we add an ioctl VFIO_IOMMU_ENABLE_IOPF to make it configurable. And the >> IOPF enablin

Re: [RFC PATCH v3 4/8] vfio/type1: Pre-map more pages than requested in the IOPF handling

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:16 +0800 > Shenming Lu wrote: > >> To optimize for fewer page fault handlings, we can pre-map more pages >> than requested at once. >> >> Note that IOPF_PREMAP_LEN is just an arbitrary val

Re: [RFC PATCH v3 3/8] vfio/type1: Add an MMU notifier to avoid pinning

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:15 +0800 > Shenming Lu wrote: > >> To avoid pinning pages when they are mapped in IOMMU page tables, we >> add an MMU notifier to tell the addresses which are no longer valid >> and try to unma

Re: [RFC PATCH v3 1/8] iommu: Evolve the device fault reporting framework

2021-05-20 Thread Shenming Lu
On 2021/5/19 2:58, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:13 +0800 > Shenming Lu wrote: > >> This patch follows the discussion here: >> >> https://lore.kernel.org/linux-acpi/YAaxjmJW+ZMvrhac@myrica/ >> >> Besides SVA/vSVA, such as VFIO may

Re: [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough

2021-05-20 Thread Shenming Lu
Hi Alex, On 2021/5/19 2:57, Alex Williamson wrote: > On Fri, 9 Apr 2021 11:44:12 +0800 > Shenming Lu wrote: > >> Hi, >> >> Requesting for your comments and suggestions. :-) >> >> The static pinning and mapping problem in VFIO and possible solutions >

Re: [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough

2021-05-11 Thread Shenming Lu
Hi Alex, Hope for some suggestions or comments from you since there seems to be many unsure points in this series. :-) Thanks, Shenming On 2021/4/26 9:41, Shenming Lu wrote: > On 2021/4/9 11:44, Shenming Lu wrote: >> Hi, >> >> Requesting for your comments and suggestion

Re: [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough

2021-04-25 Thread Shenming Lu
On 2021/4/9 11:44, Shenming Lu wrote: > Hi, > > Requesting for your comments and suggestions. :-) Kind ping... > > The static pinning and mapping problem in VFIO and possible solutions > have been discussed a lot [1, 2]. One of the solutions is to add I/O > Page Fault sup

[RFC PATCH v3 7/8] vfio/type1: Add selective DMA faulting support

2021-04-08 Thread Shenming Lu
directly return with an invalid response. Suggested-by: Kevin Tian Signed-off-by: Shenming Lu --- drivers/vfio/vfio.c | 4 +- drivers/vfio/vfio_iommu_type1.c | 357 +++- include/linux/vfio.h| 1 + 3 files changed, 358 insertions(+), 4

[RFC PATCH v3 8/8] vfio: Add nested IOPF support

2021-04-08 Thread Shenming Lu
handler (a consolidated one) via flags (set FAULT_REPORT_NESTED_L1), and further deliver the received stage 1 faults in the handler to the guest through a newly added vfio_device_ops callback. Signed-off-by: Shenming Lu --- drivers/vfio/vfio.c | 81

[RFC PATCH v3 6/8] vfio/type1: No need to statically pin and map if IOPF enabled

2021-04-08 Thread Shenming Lu
dirty tracking support in the future. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 38 +++-- 1 file changed, 32 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 7df5711e743a

[RFC PATCH v3 5/8] vfio/type1: VFIO_IOMMU_ENABLE_IOPF

2021-04-08 Thread Shenming Lu
supported since there may be inflight page faults when disabling. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 223 +++- include/uapi/linux/vfio.h | 6 + 2 files changed, 226 insertions(+), 3 deletions(-) diff --git a/drivers/vfio

[RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough

2021-04-08 Thread Shenming Lu
erative DMA Buffer Tracking for Efficient Memory Management in Direct I/O. In USENIX ATC, 2020. [3] https://patchwork.kernel.org/project/linux-arm-kernel/cover/20210401154718.307519-1-jean-phili...@linaro.org/ [4] https://github.com/Linaro/uadk Thanks, Shenming Shenming Lu (8): iommu: Evolve

[RFC PATCH v3 4/8] vfio/type1: Pre-map more pages than requested in the IOPF handling

2021-04-08 Thread Shenming Lu
To optimize for fewer page fault handlings, we can pre-map more pages than requested at once. Note that IOPF_PREMAP_LEN is just an arbitrary value for now, which we could try further tuning. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 131

[RFC PATCH v3 1/8] iommu: Evolve the device fault reporting framework

2021-04-08 Thread Shenming Lu
s not a clean way. In addition, still take VFIO as an example, in nested mode, the 1st level and 2nd level fault reporting may be configured separately and currently each device can only register one iommu dev fault handler, so we add a handler update interface for this. Signed-off-by: Shenmi

[RFC PATCH v3 2/8] vfio/type1: Add a page fault handler

2021-04-08 Thread Shenming Lu
VFIO manages the DMA mapping itself. To support IOPF (on-demand paging) for VFIO (IOMMU capable) devices, we add a VFIO page fault handler to serve the reported page faults from the IOMMU driver. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 114

[RFC PATCH v3 3/8] vfio/type1: Add an MMU notifier to avoid pinning

2021-04-08 Thread Shenming Lu
To avoid pinning pages when they are mapped in IOMMU page tables, we add an MMU notifier to tell the addresses which are no longer valid and try to unmap them. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 112 +++- 1 file changed, 109 insertions

Re: [RFC PATCH v2 1/6] iommu: Evolve to support more scenarios of using IOPF

2021-03-09 Thread Shenming Lu
Hi Baolu, On 2021/3/10 10:09, Lu Baolu wrote: > Hi Shenming, > > On 3/9/21 2:22 PM, Shenming Lu wrote: >> This patch follows the discussion here: >> >> https://lore.kernel.org/linux-acpi/YAaxjmJW+ZMvrhac@myrica/ >> >> In order to support more scenario

[RFC PATCH v2 6/6] vfio: Add nested IOPF support

2021-03-08 Thread Shenming Lu
handler (a combined one) via flags (set IOPF_REPORT_NESTED_L1_CONCERNED), and further deliver the received stage 1 faults in the handler to the guest through a newly added vfio_device_ops callback. Signed-off-by: Shenming Lu --- drivers/vfio/vfio.c | 83

[RFC PATCH v2 4/6] vfio: VFIO_IOMMU_ENABLE_IOPF

2021-03-08 Thread Shenming Lu
not supported since there may be inflight page faults when disabling. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 139 +++- include/uapi/linux/vfio.h | 6 ++ 2 files changed, 142 insertions(+), 3 deletions(-) diff --git a/drivers/vfio

[RFC PATCH v2 5/6] vfio: No need to statically pin and map if IOPF enabled

2021-03-08 Thread Shenming Lu
If IOPF enabled for the VFIO container, there is no need to statically pin and map the entire DMA range, we can do it on demand. And unmap according to the IOPF mapped bitmap when removing vfio_dma. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 35

[RFC PATCH v2 3/6] vfio: Add a page fault handler

2021-03-08 Thread Shenming Lu
VFIO manages the DMA mapping itself. To support IOPF for VFIO devices, we add a VFIO page fault handler to serve the reported page faults from the IOMMU driver. Besides, we can pre-map more pages than requested at once to optimize for fewer page fault handlings. Signed-off-by: Shenming Lu

[RFC PATCH v2 1/6] iommu: Evolve to support more scenarios of using IOPF

2021-03-08 Thread Shenming Lu
be interested in only one level. Signed-off-by: Shenming Lu --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 3 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 ++-- drivers/iommu/io-pgfault.c| 4 -- drivers/iommu/iommu.c | 56

[RFC PATCH v2 2/6] vfio: Add an MMU notifier to avoid pinning

2021-03-08 Thread Shenming Lu
To avoid pinning pages when they are mapped in IOMMU page tables, we add an MMU notifier to tell the addresses which are no longer valid and try to unmap them. Signed-off-by: Shenming Lu --- drivers/vfio/vfio_iommu_type1.c | 68 + 1 file changed, 68 insertions

[RFC PATCH v2 0/6] Add IOPF support for VFIO passthrough

2021-03-08 Thread Shenming Lu
comments and suggestions are very welcome. :-) Thanks, Shenming Shenming Lu (6): iommu: Evolve to support more scenarios of using IOPF vfio: Add an MMU notifier to avoid pinning vfio: Add a page fault handler vfio: VFIO_IOMMU_ENABLE_IOPF vfio: No need to statically pin and map if IOPF

Re: [PATCH v11 04/13] vfio/pci: Add VFIO_REGION_TYPE_NESTED region type

2021-02-23 Thread Shenming Lu
> +static int vfio_pci_dma_fault_init(struct vfio_pci_device *vdev) > +{ > + struct vfio_region_dma_fault *header; > + struct iommu_domain *domain; > + size_t size; > + bool nested; > + int ret; > + > + domain = iommu_get_domain_for_dev(&vdev->pdev->dev); > + ret = iommu

Re: [PATCH v12 06/10] iommu: Add a page fault handler

2021-02-01 Thread Shenming Lu
Hi Jean, It seems that the preprocessing of the page faults(groups) here is relatively generic, and if a device driver wants to reuse it while having its own iopf_handle_single(), is there any chance for this? :-) Thanks, Shenming On 2021/1/27 23:43, Jean-Philippe Brucker wrote: > Some systems