On Wed, Jun 12, 2019 at 3:25 AM Oded Gabbay <oded.gab...@gmail.com> wrote: > > On Tue, Jun 11, 2019 at 8:03 PM Oded Gabbay <oded.gab...@gmail.com> wrote: > > > > On Tue, Jun 11, 2019 at 6:26 PM Greg KH <gre...@linuxfoundation.org> wrote: > > > *snip* > > > > Now, when I tried to integrate Goya into a POWER9 machine, I got a > > reject from the call to pci_set_dma_mask(pdev, 48). The standard code, > > as I wrote above, is to call the same function with 32-bits. That > > works BUT it is not practical, as our applications require much more > > memory mapped then 32-bits.
Setting a 48 bit DMA mask doesn't work today because we only allocate IOMMU tables to cover the 0..2GB range of PCI bus addresses. Alexey has some patches to expand that range so we can support devices that can't hit the 64 bit bypass window. You need: This fix: http://patchwork.ozlabs.org/patch/1113506/ This series: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=110810 Give that a try and see if the IOMMU overhead is tolerable. > >In addition, once you add more cards which > > are all mapped to the same range, it is simply not usable at all. Each IOMMU group should have a separate bus address space and seperate cards shouldn't be in the same IOMMU group. If they are then there's something up. Oliver
