>-----Original Message----- >From: Joao Martins <joao.m.mart...@oracle.com> >Sent: Friday, November 10, 2023 9:09 PM >Subject: Re: [PATCH v4 28/41] vfio/iommufd: Implement the iommufd backend > >On 10/11/2023 03:15, Duan, Zhenzhong wrote: >> Hi Jason, Joao, >> >>> -----Original Message----- >>> From: Jason Gunthorpe <j...@nvidia.com> >>> Sent: Thursday, November 9, 2023 10:35 PM >>> Subject: Re: [PATCH v4 28/41] vfio/iommufd: Implement the iommufd >backend >>> >>> On Thu, Nov 09, 2023 at 01:21:59PM +0000, Joao Martins wrote: >>>> On 09/11/2023 13:09, Jason Gunthorpe wrote: >>>>> On Thu, Nov 09, 2023 at 01:03:02PM +0000, Joao Martins wrote: >>>>> >>>>>>> I am not talking about mdevs; but rather the regular (non mdev) case not >>> being >>>>>>> able to use dirty tracking with autodomains hwpt allocation. >>>>>> >>>>>> ... without any vIOMMU. >>>>> >>>>> Ah, well, that is troublesome isn't it.. >>>>> >>>>> So do we teach autodomains to be more featured in the kernel or do we >>>>> teach the generic qemu code to effectively implement autodomains in >>>>> userspace? >>>> >>>> The latter is actually what we have been doing. Well I wouldn't call >>> autodomains >>>> in qemu, but rather just allocate a hwpt, instead of attaching the IOAS >>>> directly. But well mdevs don't have domains and we overlooked that. I would >>> turn >>>> the exception into an exception rather than making the norm, doesn't look >>>> to >>> be >>>> much complexity added? >>> >>> Autodomains are complex because of things like mdev and iommu >>> non-uniformity's. Qemu can't just allocate a single HWPT, it needs to >>> be annoyingly managed. >>> >>>> What I last re-collect is that autodomains represents the 'simple users' >>>> that >>>> don't care much beyond the basics of IOMMU features (I recall the example >>> was >>>> DPDK apps and the like). You could say that for current needs IOMMU >>> autodomains >>>> suffices for qemu. >>> >>> Yes, that was my intention. Aside from that it primarily exists to >>> support vfio compatibility >>> >>>> Connecting autodomains to this enforcing on the hwpt is relatively simple >btw, >>>> it just needs to connect the dirty tracking flag with same semantic of >>>> hwpt-alloc equivalent and pass the hwpt flags into the domain allocation. >>> >>> Yes >>> >>>> It's more of what of a question should be the expectations to the user when >>>> using ATTACH_HWPT with an IOAS_ID versus direct manipulation of HWPT. I >am >>>> wondering if dirty tracking is alone here or whether there's more features >that >>>> start to mud the simplicity of autodomains that would approximate of hwpt- >>> alloc. >>> >>> This is why I had been thinking of a pure HWPT based scheme >>> >>> So it seems we cannot have a simple model where the generic qmeu layer >>> just works in IOAS :( It might as well always work in HWPT and >>> understand all the auto domains complexity itself. >> >> Let me know if there is anything I can do in this series to facilitate >> future qemu dirty tracking support of iommufd. Not clear if I should >> restore to the manual HWPT_ALLOC method in v4. > >If we want to have the closest support as type1-iommu, from what we have been >discussing... it sounds like IOAS is the easiest first step to get barebones >iommufd support. Which sort of makes sense since this is the introduction of >iommufd and it already requires a lot of churn & refactoring to get there. Agree.
> >For the new iommufd-only features (nesting/dirty-tracking) we will need the >auto >domains done by Qemu IIUC -- unless nesting is meant to coexist with >autodomains >with its own hwpts somehow (?) We have a draft nesting implementation which has its own hwpts and coexist with autodomains. See https://github.com/yiliu1765/qemu/tree/zhenzhong/iommufd_cdev_v5_nesting if you are interested. > >Right now I don't have the autodomains QEMU equivalent structure in mind to >suggest a path in alternative to v5; Looking at the kernel autodomains path, >aside from mdev I am not sure yet what annoyances the autodomains path in >qemu >is going to generate: more worringly whether we have enough information to >tackle the non-uniformity e.g. if we are talking about features or whether >different devices are behind different IOMMUs. OK, looks more thinking and discuss needed except mdev. I'd like to keep this series as a basic iommufd support with IOAS attaching. QEMU autodomain may be another series addressing new iommufd-only features. Thanks Zhenzhong
