On Thu, May 29, 2025 at 03:38:15PM -0300, Jason Gunthorpe wrote:
> On Thu, May 29, 2025 at 11:22:19AM -0700, Shyam Saini wrote:
> > > All IOVA that the platform cannot DMA from should be reported in the
> > > reserved_regions file as "reserved". You must make your platform
> > > achieve this.
> >
> > so should it be for all the iommu groups?
> >
> > no_dma_mem {
> > reg = <0x0 0x8000000 0x0 0x100000>;
> > no-map;
> > };
> >
> > i think that's how we reserve memory in general, eg: ramoops
> > but this doesn't show up in:
> > /sys/kernel/iommu_groups/*/reserved_regions
>
> I don't know anymore, I've forgotten these details about DT, you will
> need to consult with DT experts..
no worries, added maintainers and the list
> > > But I still don't have a clear sense of what your actual problem is as
> > > you are show DT that seems reasonable and saying that
> > > /sys/../reserved_regions is working..
> >
> > /sys/../reserved_regions is working for certain devices like dma controller
> > but it doesn't work for pcie devices and its vfio-pcie driver calling
> > iommu_get_resv_regions() but we don't have dts node for vfio.
> > I have confirmed this about pcie on two different platforms, it seems to be
> > OF DMA-API gap that you hinted above, happy to work on that :), it would be
> > great if you can share any other reference discussions to that problem
>
> Some of it was around the dma_mask and the ranges, but I'm not sure
> exactly what is your issue here..
>
> It is very important the the dma-iommu.c code also knows not to use
> the non-working IOVA or your system will be buggy and eventually fail.
> If that isn't happening you have a critical bug beyond the SW_MSI
> issue.
thanks, i will look into this
> The easiest way to fix it is to have reserved_regions report the right
> stuff.
>
> > When i specify this for dma controller:
> >
> > faulty_iova: resv_faulty {
> > iommu-addresses = <&dmaX 0x8000000 0x100000>;
> > };
> > &dmaX {
> > memory-region = <&faulty_iova>;
> > };
> >
> > I see following:
> > $ cat /sys/kernel/iommu_groups/y/reserved_regions
> > 0x0000000008000000 0x00000000080fffff reserved
> > 0x00000000a0000000 0x00000000a00fffff msi
>
> See, that looks better. You need to make that happen for all the
> effected devices.
>
> > Upon investigation, our hardware team confirmed that the memory region
> > containing 0x08000000 is already mapped for other peripherals, making it
> > unavailable for MSI usage.
>
> See, that's just wrong HW design. The path from PCI to the SMMU should
> not have any other decodes on it, isn't that SBSA?
I am not 100% sure
> How does your platform work at all? Isn't 0x08000000 physical memory in
> your address map?
unfornately, this 0x08000000 physical address is not in our address map
Thanks,
Shyam
