> -----Original Message-----
> From: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
> Sent: Thursday, May 30, 2024 10:01 PM
> To: Liu, Yuan1 <yuan1....@intel.com>; pet...@redhat.com; faro...@suse.de
> Cc: qemu-devel@nongnu.org; Linuxarm <linux...@huawei.com>; linwenkai (C)
> <linwenk...@hisilicon.com>; zhangfei....@linaro.org; huangchenghai
> <huangchengh...@huawei.com>
> Subject: RE: [PATCH 1/7] docs/migration: add uadk compression feature
>
>
>
> > -----Original Message-----
> > From: Liu, Yuan1 <yuan1....@intel.com>
> > Sent: Thursday, May 30, 2024 2:25 PM
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>;
> > pet...@redhat.com; faro...@suse.de
> > Cc: qemu-devel@nongnu.org; Linuxarm <linux...@huawei.com>; linwenkai (C)
> > <linwenk...@hisilicon.com>; zhangfei....@linaro.org; huangchenghai
> > <huangchengh...@huawei.com>
> > Subject: RE: [PATCH 1/7] docs/migration: add uadk compression feature
> >
> > > -----Original Message-----
> > > From: Shameer Kolothum <shameerali.kolothum.th...@huawei.com>
> > > Sent: Wednesday, May 29, 2024 5:44 PM
> > > To: pet...@redhat.com; faro...@suse.de; Liu, Yuan1
> <yuan1....@intel.com>
> > > Cc: qemu-devel@nongnu.org; linux...@huawei.com;
> > linwenk...@hisilicon.com;
> > > zhangfei....@linaro.org; huangchengh...@huawei.com
> > > Subject: [PATCH 1/7] docs/migration: add uadk compression feature
>
> [...]
>
> > > +Since UADK uses Shared Virtual Addressing(SVA) and device access
> virtual
> > > memory
> > > +directly it is possible that SMMUv3 may enounter page faults while
> > > walking the
> > > +IO page tables. This may impact the performance. In order to mitigate
> > > this,
> > > +please make sure to specify ``-mem-prealloc`` parameter to the
> > > destination VM
> > > +boot parameters.
> >
> > Thank you so much for putting the IAA solution at the top and cc me.
> >
> > I think migration performance will be better with '-mem-prealloc'
> option,
> > but I am considering whether '-mem-prealloc' is a mandatory option, from
> my
> > experience, SVA performance drops mainly caused by IOTLB flush and IO
> page
> > fault,
> > I had some discussions with Peter Xu about the IOTLB flush issue, and it
> has
> > been improved.
> > https://patchew.org/QEMU/PH7PR11MB5941F04FBFB964CB2C968866A33E2@
> > PH7PR11MB5941.namprd11.prod.outlook.com/
>
> Thanks for the link. Yes I have seen that discussion and this series is on
> top of that
> patch for avoiding the zero page read fault.
>
> >
> > For IO page fault, the QPL(IAA userspace library) can process page fault
> > request instead of IOMMU,
>
> Sorry I didn't get this part completely. So if the page fault happens how
> the library
> can handle it without IOMMU? Or you meant library will do memory
> perfecting before
> to avoid the page fault?
Yes, when the I/O page fault happens, the hardware will return the fault address
to the QPL, QPL will populate the memory as below, then resubmit the job to
hardware again.
if (AD_STATUS_READ_PAGE_FAULT == completion_record_ptr->status) {
volatile char* read_fault_address = (char *)fault_address;
*read_fault_address;
}
else { // AD_STATUS_WRITE_PAGE_FAULT
volatile char* write_fault_address = (char *)fault_address;
*write_fault_address = *write_fault_address;
}
> it means we can disable the I/O page fault feature
> > on the IAA device, and let the device still use SVA technology to avoid
> memory
> > copy.
> >
> > I will provide the test results in my next version, do you have any
> ideas or
> > suggestions about this, thanks.
>
> I think our UADK test tool had an option to prefect the memory(write some
> random data
> to memory) to avoid page fault penalty. I am not sure that is exposed
> through the API or not.
> I will check with our UADK team.
>
> Please do CC me when you post your next revision.
Sure
> Thanks,
> Shameer
