On Tue, Nov 05, 2019 at 04:36:59PM +0100, Olivier Matz wrote:
> KNI supposes that mbufs are contiguous in kernel virtual memory. This
> may not be true when using the IOVA=VA mode. To fix this, a possibility
> is to ensure that objects do not cross page boundaries in mempool. This
> patchset implements this in the last patch (5/5).
>
> The previous patches prepare the job:
> - allow to populate with an unaligned virtual area (1/5).
> - reduce spaced wasted in the mempool size calculation when not using
> the iova-contiguous allocation (2/5).
> - remove the iova-contiguous allocation when populating mempool (3/5):
> a va-contiguous alloc does the job as well if we want to populate
> without crossing page boundaries, so simplify the mempool populate
> function.
> - export a function to get the minimum page used in a mempool (4/5)
>
> Memory consumption impact when using hugepages:
> - worst case: + ~0.1% for a mbuf pool (objsize ~= 2368)
> - best case: -50% for if pool size is just above page size
>
> The memory consumption impact with 4K pages in IOVA=VA mode could
> however consume up to 75% more memory for mbuf pool, because there will
> be only 1 mbuf per page. Not sure how common this usecase is.
>
> Caveat: this changes the behavior of the mempool (calc_mem_size and
> populate), and there is a small risk to break things, especially with
> alternate mempool drivers.
>
> v4
>
> * remove useless comments in Makefiles and meson.build (sugg by David)
> * add EXPERIMENTAL banner on new functions in API comments (David)
> * sort by version in rte_mempool_version.map (David)
> * remove duplicated -DALLOW_EXPERIMENTAL_API flag in octeontx2
> mempool driver
> * enhance API comments for new helpers
I forgot:
* move bucket mempool driver modifications from patch 7 to
patch 6 (seen by Andrew)
>
> v3
>
> * introduce new helpers to calculate required memory size and to
> populate mempool, use them in drivers: the alignment constraint
> of octeontx/octeontx2 is managed in this common code.
> * fix octeontx mempool driver by taking alignment constraint in account
> like in octeontx2
> * fix bucket mempool driver with 4K pages: limit bucket size in this
> case to ensure that objects do not cross page boundaries. With larger
> pages, it was already ok, because bucket size (64K) is smaller than
> a page.
> * fix some api comments in mempool header file
>
> v2
>
> * update octeontx2 driver to keep alignment constraint (issue seen by
> Vamsi)
> * add a new patch to use RTE_MEMPOOL_ALIGN (Andrew)
> * fix initialization of for loop in rte_mempool_populate_virt() (Andrew)
> * use rte_mempool_populate_iova() if mz_flags has
> RTE_MEMZONE_IOVA_CONTIG (Andrew)
> * check rte_mempool_get_page_size() return value (Andrew)
> * some other minor style improvements
>
> rfc -> v1
>
> * remove first cleanup patch, it was pushed separately
> a2b5a8722f20 ("mempool: clarify default populate function")
> * add missing change in rte_mempool_op_calc_mem_size_default()
> * allow unaligned addr/len in populate virt
> * better split patches
> * try to better explain the change
> * use DPDK align macros when relevant
>
> Olivier Matz (7):
> mempool: allow unaligned addr/len in populate virt
> mempool: reduce wasted space on mempool populate
> mempool: remove optimistic IOVA-contiguous allocation
> mempool: introduce function to get mempool page size
> mempool: introduce helpers for populate and calc mem size
> mempool: prevent objects from being across pages
> mempool: use the specific macro for object alignment
>
> drivers/mempool/bucket/Makefile | 1 +
> drivers/mempool/bucket/meson.build | 2 +
> drivers/mempool/bucket/rte_mempool_bucket.c | 10 +-
> drivers/mempool/dpaa/dpaa_mempool.c | 4 +-
> drivers/mempool/dpaa2/dpaa2_hw_mempool.c | 4 +-
> drivers/mempool/octeontx/Makefile | 2 +
> drivers/mempool/octeontx/meson.build | 2 +
> .../mempool/octeontx/rte_mempool_octeontx.c | 21 +--
> drivers/mempool/octeontx2/Makefile | 2 +
> drivers/mempool/octeontx2/meson.build | 2 +
> drivers/mempool/octeontx2/otx2_mempool_ops.c | 21 ++-
> lib/librte_mempool/rte_mempool.c | 147 +++++++-----------
> lib/librte_mempool/rte_mempool.h | 114 ++++++++++++--
> lib/librte_mempool/rte_mempool_ops.c | 4 +-
> lib/librte_mempool/rte_mempool_ops_default.c | 113 +++++++++++---
> lib/librte_mempool/rte_mempool_version.map | 6 +
> 16 files changed, 312 insertions(+), 143 deletions(-)
>
> --
> 2.20.1
>
