POWER9 (PHB4) requires all peripherals using DMA to be either restricted to 32-bit windows or capable of accessing the entire 64 bits of memory space. Some devices, such as most GPUs, can only address up to a certain number of bits (approximately 40, in many cases), while at the same time it is highly desireable to use a larger DMA space than the fallback 32 bits.
This series adds something called "cognitive DMA", which is a form of dynamic TCE allocation. This allows the peripheral to DMA to host addresses mapped in 1G (PHB4) or 256M (PHB3) chunks, and is transparent to the peripheral and its driver stack. This series has been tested on a Talos II server with a Radeon WX4100 and a wide range of OpenGL applications. While there is still work, notably involving what happens if a peripheral attempts to DMA close to a TCE window boundary, this series greatly improves functionality for AMD GPUs on POWER9 devices over the existing 32-bit DMA support. Russell Currey (4): powerpc/powernv/pci: Track largest available TCE order per PHB powerpc/powernv: DMA operations for discontiguous allocation powerpc/powernv/pci: Track DMA and TCE tables in debugfs powerpc/powernv/pci: Safety fixes for pseudobypass TCE allocation Timothy Pearson (3): powerpc/powernv/pci: Export pnv_pci_ioda2_tce_invalidate_pe powerpc/powernv/pci: Invalidate TCE cache after DMA map setup powerpc/powernv/pci: Don't use the lower 4G TCEs in pseudo-DMA mode arch/powerpc/include/asm/dma-mapping.h | 1 + arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/pci-dma.c | 320 ++++++++++++++++++++++ arch/powerpc/platforms/powernv/pci-ioda.c | 169 ++++++++---- arch/powerpc/platforms/powernv/pci.h | 11 + 5 files changed, 452 insertions(+), 51 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/pci-dma.c -- 2.17.1
