Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Tue, Jun 08, 2021 at 04:04:06PM -0300, Jason Gunthorpe wrote: > On Tue, Jun 08, 2021 at 10:53:02AM +1000, David Gibson wrote: > > On Thu, Jun 03, 2021 at 08:52:24AM -0300, Jason Gunthorpe wrote: > > > On Thu, Jun 03, 2021 at 03:13:44PM +1000, David Gibson wrote: > > &

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Thu, Jun 10, 2021 at 06:37:31PM +0200, Jean-Philippe Brucker wrote: > On Tue, Jun 08, 2021 at 04:31:50PM +1000, David Gibson wrote: > > For the qemu case, I would imagine a two stage fallback: > > > > 1) Ask for the exact IOMMU capabilities (including pagetable > &

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Thu, Jun 03, 2021 at 08:12:27AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, June 2, 2021 2:15 PM > > > [...] > > > > > > > /* > > > * Get information about an I/O address space > > > * > > &g

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
ch. "Good" modern devices on modern systems will be both fully isolated and well identified, so for the uses cases that people seem to mostly care about here we'll still have identification group == isolation group == one device. In other words, do we really have

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Tue, Jun 08, 2021 at 10:17:56AM -0300, Jason Gunthorpe wrote: > On Tue, Jun 08, 2021 at 12:37:04PM +1000, David Gibson wrote: > > > > The PPC/SPAPR support allows KVM to associate a vfio group to an IOMMU > > > page table so that it can handle iotlb programming from pre

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
ow do I use PASID functionality of a device behind a > !ACS switch if the uAPI forces all IOASID's to be linked to a group, > not a device? > > Device centric with an report that "all devices in the group must use > the same IOASID" covers all the new functionality, k

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
IOASIDs. That means that if we allow attaching devices within a group to different IOASIDs we effectively need to introduce two levels of "group-like" things. First the idenfication group, then the isolation group. You're using "group" for the isolation group, but then we ha

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
olated by their mdev driver are in a different group from their parent device, and therefore need not be affected by whether the parent device shares a group with some other physical device. They *might* be, but that's up to the mdev driver to determine based on what it can safely isolate.

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Wed, Jun 23, 2021 at 08:00:49AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, June 17, 2021 12:08 PM > > > > On Thu, Jun 03, 2021 at 08:12:27AM +, Tian, Kevin wrote: > > > > From: David Gibson > >

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Thu, Jun 17, 2021 at 08:10:04PM -0300, Jason Gunthorpe wrote: > On Thu, Jun 17, 2021 at 02:45:46PM +1000, David Gibson wrote: > > On Wed, Jun 09, 2021 at 09:39:19AM -0300, Jason Gunthorpe wrote: > > > On Wed, Jun 09, 2021 at 02:24:03PM +0200, Joerg Roedel wrote: > > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Fri, Jun 18, 2021 at 07:03:31PM +0200, Jean-Philippe Brucker wrote: > On Thu, Jun 17, 2021 at 01:00:14PM +1000, David Gibson wrote: > > On Thu, Jun 10, 2021 at 06:37:31PM +0200, Jean-Philippe Brucker wrote: > > > On Tue, Jun 08, 2021 at 04:31:50PM +1000, David Gibson wrote: &g

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Fri, Jun 18, 2021 at 01:21:47PM +0800, Lu Baolu wrote: > Hi David, > > On 6/17/21 1:22 PM, David Gibson wrote: > > > The iommu_group can guarantee the isolation among different physical > > > devices (represented by RIDs). But when it comes to sub-devices (ex. mdev

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Wed, Jun 23, 2021 at 07:59:21AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, June 17, 2021 11:48 AM > > > > On Tue, Jun 08, 2021 at 10:17:56AM -0300, Jason Gunthorpe wrote: > > > On Tue, Jun 08, 2021 at 12:37:04PM +1000, David Gibs

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
the domain of another device, unavoidably. It may be rare with modern hardware, but we still can't ignore the case. Which means you can't DMA block until everything in the group is managed by a vfio-like driver. -- David Gibson| I'll have my music baroqu

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
efore a container must exist in a single > address space in the PCI topology. In a conventional or non-vIOMMU > topology, the PCI address space is equivalent to the system memory > address space. When vIOMMU gets involved, multiple devices within the > same group must exist in the same

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Thu, Jun 17, 2021 at 08:04:38PM -0300, Jason Gunthorpe wrote: > On Thu, Jun 17, 2021 at 03:02:33PM +1000, David Gibson wrote: > > > In other words, do we really have use cases where we need to identify > > different devices IDs, even though we know they're not isolated.

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
value to add new features on it)? Then all new > sub-systems including vdpa and new vfio only support singleton > device group via /dev/iommu... The case that worries me here is if you *thought* you had 1 device groups, but then discover a hardware bug which means two t

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
sn't need to know the > attaching status of each device in a group before it can allow user to > access a device. As long as a device in a group either in block DMA > or switch to a new address space created via /dev/iommu FD, there's > no problem to allow user acc

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-25 Thread David Gibson
SID capable dev (option 2) IOAS G1 IOPT format: Kernel managed DMA address type: 64-bit Parent DMA address type: Root (User Virtual Address) => one-PASID endpoint for device DMA address type: 64-bit DMA identifier type: RID+PASID Driver makes mapp

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-02 Thread David Gibson
On Fri, Jul 30, 2021 at 11:51:23AM -0300, Jason Gunthorpe wrote: > On Mon, Jul 26, 2021 at 02:50:48PM +1000, David Gibson wrote: > > > That said, I'm still finding the various ways a device can attach to > > an ioasid pretty confusing. Here are some thoughts on some extra

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-02 Thread David Gibson
On Wed, Jul 28, 2021 at 04:04:24AM +, Tian, Kevin wrote: > Hi, David, > > > From: David Gibson > > Sent: Monday, July 26, 2021 12:51 PM > > > > On Fri, Jul 09, 2021 at 07:48:44AM +, Tian, Kevin wrote: > > > /dev/iommu provides an unified int

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
est according to its vIOMMU semantics. Then SVA capable devices would be plugged into that IOAS by using "PASID+address" type endpoints from those devices. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalis

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
On Wed, Aug 04, 2021 at 11:07:42AM -0300, Jason Gunthorpe wrote: > On Tue, Aug 03, 2021 at 11:58:54AM +1000, David Gibson wrote: > > > I'd rather deduce the endpoint from a collection of devices than the > > > other way around... > > > > Which I think is co

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
On Tue, Aug 03, 2021 at 03:19:26AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Tuesday, August 3, 2021 9:51 AM > > > > On Wed, Jul 28, 2021 at 04:04:24AM +, Tian, Kevin wrote: > > > Hi, David, > > > > > > > From: Davi

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-09 Thread David Gibson
On Mon, Aug 09, 2021 at 08:34:06AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Friday, August 6, 2021 12:45 PM > > > > > In concept I feel the purpose of DMA endpoint is equivalent to the > > routing > > > > > info in this proposal.

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-10 Thread David Gibson
On Fri, Aug 06, 2021 at 09:32:11AM -0300, Jason Gunthorpe wrote: > On Fri, Aug 06, 2021 at 02:45:26PM +1000, David Gibson wrote: > > > Well, that's kind of what I'm doing. PCI currently has the notion of > > "default" address space for a RID, but there&#

Re: [RFC 04/20] iommu: Add iommu_device_get_info interface

2021-09-28 Thread David Gibson
iommu_devattr attr, void > *data); > + > #else /* CONFIG_IOMMU_API */ > > struct iommu_ops {}; > @@ -999,6 +1012,12 @@ static inline struct iommu_fwspec > *dev_iommu_fwspec_get(struct device *dev) > { > return NULL; > } > + > +static inline int iommu_device_get

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-28 Thread David Gibson
think you'll make things much easier if instead of using one huge "devices" subdir, you use a separate subdir for each vfio sub-driver (so, one for PCI, one for each type of mdev, one for platform, etc.). That should make avoiding name conflicts a lot simpler. -

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-28 Thread David Gibson
fio.device_lock); > idr_init(&vfio.device_idr); > + INIT_LIST_HEAD(&vfio.device_list); > > /* /dev/vfio/devices/$DEVICE */ > vfio.device_class = class_create(THIS_MODULE, "vfio-device"); > @@ -2542,6 +2654,7 @@ static int __init vfio_init(void) &g

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-28 Thread David Gibson
> +}; > + > +/* > + * An iommu group is viable for use by userspace if all devices are in > + * one of the following states: > + * - driver-less > + * - bound to an allowed driver > + * - a PCI interconnect device > + */ > +static int device_user_dma_viable(struct dev

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-28 Thread David Gibson
MMU_DEVICE_INFO_ADDR_WIDTH (1 << 2) /* addr_wdith field valid */ > + __u64 dev_cookie; > + __u64 pgsize_bitmap; > + __u32 addr_width; I think this is where you should be reporting available IOVA windows, rather than just an address width. I know that for ppc a re

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-28 Thread David Gibson
ice cookie when calling this ioctl. The > + * cookie is later used in iommufd for capability query, iotlb invalidation > + * and I/O fault handling. > + * > + * User is not allowed to access the device before the binding operation > + * is completed. > + * > + * Unbind is automatically conducted when device fd is closed. > + *

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-28 Thread David Gibson
e/linux/iommufd.h > @@ -0,0 +1,38 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * IOMMUFD API definition > + * > + * Copyright (C) 2021 Intel Corporation > + * > + * Author: Liu Yi L > + */ > +#ifndef __LINUX_IOMMUFD_H > +#define __LINUX_IOMMUFD_H > +

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 05:38:56AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 12:56 PM > > > > > > > > Unlike vfio, iommufd adopts a device-centric design with all group > > > logistics hidden behi

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 01:05:21PM -0600, Alex Williamson wrote: > On Wed, 29 Sep 2021 12:08:59 +1000 > David Gibson wrote: > > > On Sun, Sep 19, 2021 at 02:38:30PM +0800, Liu Yi L wrote: > > > This patch introduces a new interface (/dev/vfio/devices/$DEVICE) for > >

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 06:41:00AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 2:01 PM > > > > On Sun, Sep 19, 2021 at 02:38:36PM +0800, Liu Yi L wrote: > > > This patch adds VFIO_DEVICE_BIND_IOMMUFD for userspace to

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 09:57:16AM -0300, Jason Gunthorpe wrote: > On Wed, Sep 29, 2021 at 04:35:19PM +1000, David Gibson wrote: > > > Yes, exactly. And with a group interface it's obvious it has to > > understand it. With the non-group interface, you can get to this &g

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 09:24:57AM -0300, Jason Gunthorpe wrote: 65;6402;1c> On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > > > > +struct iommufd_device { > > > + unsigned int id; > > > + struct iommufd_ctx *ictx; > > > + struct dev

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 07:31:08AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 2:35 PM > > > > On Wed, Sep 29, 2021 at 05:38:56AM +, Tian, Kevin wrote: > > > > From: David Gibson > > > &

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-30 Thread David Gibson
On Thu, Sep 30, 2021 at 07:28:18PM -0300, Jason Gunthorpe wrote: > On Thu, Sep 30, 2021 at 01:09:22PM +1000, David Gibson wrote: > > > > The *admin* the one responsible to understand the groups, not the > > > applications. The admin has no idea what a group FD is - they sh

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-30 Thread David Gibson
ble > + * format between device and IOASID will lead to attaching failure in > + * device side. > + * > + * Currently only one flag (IOMMU_IOASID_ENFORCE_SNOOP) is supported and > + * must be always set. > + * > + * Only one I/O page table type (kernel-managed) is supporte

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-30 Thread David Gibson
rn the > actual geometry including any holes via a query. So part of the point of specifying start/end addresses is that explicitly querying holes shouldn't be necessary: if the requested range crosses a hole, it should fail. If you didn't really need all that range, you shouldn't hav

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-10-06 Thread David Gibson
On Fri, Oct 01, 2021 at 09:43:22AM -0300, Jason Gunthorpe wrote: > On Thu, Sep 30, 2021 at 01:10:29PM +1000, David Gibson wrote: > > On Wed, Sep 29, 2021 at 09:24:57AM -0300, Jason Gunthorpe wrote: > > > On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > &

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-10-10 Thread David Gibson
On Thu, Oct 07, 2021 at 08:35:03AM -0300, Jason Gunthorpe wrote: > On Thu, Oct 07, 2021 at 12:23:13PM +1100, David Gibson wrote: > > On Fri, Oct 01, 2021 at 09:43:22AM -0300, Jason Gunthorpe wrote: > > > On Thu, Sep 30, 2021 at 01:10:29PM +1000, David Gibson wrote: > > >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-11 Thread David Gibson
On Fri, Oct 01, 2021 at 09:22:25AM -0300, Jason Gunthorpe wrote: > On Fri, Oct 01, 2021 at 04:13:58PM +1000, David Gibson wrote: > > On Tue, Sep 21, 2021 at 02:44:38PM -0300, Jason Gunthorpe wrote: > > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > &g

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-13 Thread David Gibson
up or not? Why does one deserve a warning and one not? > WARN_ON(1); > goto out_unlock; > } > @@ -3368,6 +3384,7 @@ static int iommu_group_init_user_dma(struct iommu_group > *group, > > group->user_dma_owne

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-13 Thread David Gibson
On Mon, Oct 11, 2021 at 03:49:14PM -0300, Jason Gunthorpe wrote: > On Mon, Oct 11, 2021 at 05:02:01PM +1100, David Gibson wrote: > > > > This means we cannot define an input that has a magic HW specific > > > value. > > > > I'm not entirely sure what yo

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-13 Thread David Gibson
On Mon, Oct 11, 2021 at 09:49:57AM +0100, Jean-Philippe Brucker wrote: > On Mon, Oct 11, 2021 at 05:02:01PM +1100, David Gibson wrote: > > qemu wants to emulate a PAPR vIOMMU, so it says (via interfaces yet to > > be determined) that it needs an IOAS where things can be mapped in

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-13 Thread David Gibson
On Wed, Oct 13, 2021 at 07:00:58AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Friday, October 1, 2021 2:11 PM > > > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > > This patch adds IOASID allocation/free interface per iommufd

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-17 Thread David Gibson
On Thu, Oct 14, 2021 at 07:06:07AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, October 14, 2021 1:24 PM > > > > On Sun, Sep 19, 2021 at 02:38:41PM +0800, Liu Yi L wrote: > > > From: Lu Baolu > > > > > > These t

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-17 Thread David Gibson
On Thu, Oct 14, 2021 at 11:52:08AM -0300, Jason Gunthorpe wrote: > On Thu, Oct 14, 2021 at 03:53:33PM +1100, David Gibson wrote: > > > > My feeling is that qemu should be dealing with the host != target > > > case, not the kernel. > > > > > > The kernel

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-24 Thread David Gibson
On Mon, Oct 18, 2021 at 01:32:38PM -0300, Jason Gunthorpe wrote: > On Mon, Oct 18, 2021 at 02:57:12PM +1100, David Gibson wrote: > > > The first user might read this. Subsequent users are likely to just > > copy paste examples from earlier things without fully understanding >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-24 Thread David Gibson
On Thu, Oct 14, 2021 at 06:53:01AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, October 14, 2021 1:00 PM > > > > On Wed, Oct 13, 2021 at 07:00:58AM +, Tian, Kevin wrote: > > > > From: David Gibson > > > > Sent: Friday

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-25 Thread David Gibson
On Mon, Oct 25, 2021 at 09:14:10AM -0300, Jason Gunthorpe wrote: > On Mon, Oct 25, 2021 at 04:14:56PM +1100, David Gibson wrote: > > On Mon, Oct 18, 2021 at 01:32:38PM -0300, Jason Gunthorpe wrote: > > > On Mon, Oct 18, 2021 at 02:57:12PM +1100, David Gibson wrote: > > &

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-26 Thread David Gibson
On Mon, Oct 25, 2021 at 08:36:02PM -0300, Jason Gunthorpe wrote: > On Tue, Oct 26, 2021 at 12:16:43AM +1100, David Gibson wrote: > > If you attach devices A and B (both in group X) to IOAS 1, then detach > > device A, what happens? Do you detach both devices? Or do you have a >

Re: [RFC 20/20] Doc: Add documentation for /dev/iommu

2021-10-29 Thread David Gibson
+ > +When a binding operation is requested by the user, the passthrough > +framework should call iommufd_bind_device(). When the device fd is > +closed by the user, iommufd_unbind_device() should be called > +automatically:: > + > + struct iommufd_device * > + iommufd_bin

<    1   2