We propose to add a mempool flag MEMPOOL_F_NON_IO to mark pools of objects that will not be used with device IO and their memory for DMA. This will allow saving IOMMU entries by not mapping the memory used by such pools.
Immediate use case is MLX5 PMD. The hardware has its internal IOMMU where PMD registers the memory. On the data path, PMD translates VA into a key consumed by the device IOMMU. It is impractical for the PMD to register all allocated memory because of increased lookup cost both in HW and SW. Most often mbuf memory comes from mempools, so if PMD tracked them, it could almost always have mbuf memory registered before an mbuf hits the PMD. The new flag would prevent the PMD for registering memory that will never need it. Tracking the mempools and dealing with them in MLX5 PMD is the next step after the proposed change. A possible use case is IOMMU management in EAL. Mempool could translate the new flag to a hint to the memory manager, which would use it to skip adding IOMMU entries in some cases. It was considered to add MEMPOOL_F_IO with the opposite meaning. It would be automatically set for pktmbuf pools; user would be able to set it for other pools. However, current assumption is that all DPDK memory is DMA-able, it is controversial to have a flag asserting this fact.
