MLX5 hardware has its internal IOMMU where PMD registers the memory. On the data path, PMD translates VA into a key consumed by the device IOMMU. It is impractical for the PMD to register all allocated memory because of increased lookup cost both in HW and SW. Most often mbuf memory comes from mempools, so if PMD tracks them, it can almost always have mbuf memory registered before an mbuf hits the PMD. This patchset adds such tracking in the PMD and internal API to support it.
Please see [1] for the discussion of the patch 2/4 and how it can be useful outside of the MLX5 PMD. [1]: http://inbox.dpdk.org/dev/ch0pr12mb509112fadb778ab28af3771db9...@ch0pr12mb5091.namprd12.prod.outlook.com/ v5: 1. Change non-IO flag inference + various fixes (Andrew). 1. Fix callback unregistration from secondary processes (Olivier). 2. Support non-IO flag in proc-dump (David). 3. Fix the usage of locks (Olivier). 4. Avoid resource leaks in unit test (Olivier). v4: (Andrew) 1. Improve mempool event callbacks unit tests and documentation. 2. Make MEMPOOL_F_NON_IO internal and automatically inferred. Add unit tests for the inference logic. v3: Improve wording and naming; fix typos (Thomas). v2 (internal review and testing): 1. Change tracked mempool event from being created (CREATE) to being fully populated (READY), which is the state PMD is interested in. 2. Unit test the new mempool callback API. 3. Remove bogus "error" messages in normal conditions. 4. Fixes in PMD. Dmitry Kozlyuk (4): mempool: add event callbacks mempool: add non-IO flag common/mlx5: add mempool registration facilities net/mlx5: support mempool registration app/proc-info/main.c | 4 +- app/test/test_mempool.c | 360 +++++++++++++++ doc/guides/nics/mlx5.rst | 13 + doc/guides/rel_notes/release_21_11.rst | 9 + drivers/common/mlx5/mlx5_common_mp.c | 50 +++ drivers/common/mlx5/mlx5_common_mp.h | 14 + drivers/common/mlx5/mlx5_common_mr.c | 580 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_mr.h | 17 + drivers/common/mlx5/version.map | 5 + drivers/net/mlx5/linux/mlx5_mp_os.c | 44 ++ drivers/net/mlx5/linux/mlx5_os.c | 4 +- drivers/net/mlx5/mlx5.c | 152 +++++++ drivers/net/mlx5/mlx5.h | 10 + drivers/net/mlx5/mlx5_mr.c | 120 ++--- drivers/net/mlx5/mlx5_mr.h | 2 - drivers/net/mlx5/mlx5_rx.h | 21 +- drivers/net/mlx5/mlx5_rxq.c | 13 + drivers/net/mlx5/mlx5_trigger.c | 77 +++- drivers/net/mlx5/windows/mlx5_os.c | 1 + lib/mempool/rte_mempool.c | 134 ++++++ lib/mempool/rte_mempool.h | 64 +++ lib/mempool/version.map | 8 + 22 files changed, 1585 insertions(+), 117 deletions(-) -- 2.25.1