From: Mikhail Mamatov <[email protected]>
Vectorized Rx keeps MLX5_VPMD_DESCS_PER_LOOP entries of elts[] pointing
to rxq->fake_mbuf after the allocated mbufs. Both rxq_cq_process_v() and
rxq_cq_decompress_v() write mbuf metadata for full SIMD groups. In
particular, CQE decompression may write through these guard entries
when processing a partial trailing group.
Both mlx5_rx_replenish_bulk_mbuf() and
mlx5_rx_mprq_replenish_bulk_mbuf() pass the guard slots directly to
rte_mempool_get_bulk(). Since a2833ecc5ea4, the mempool get path copies
available objects from the per-lcore cache to the output array before
fetching the remaining objects from the backend. If the cache contains
fewer objects than requested and the backend cannot supply the rest,
the operation restores the cache length but leaves the output array
partially overwritten.
Returning from replenishment without restoring the guards therefore
leaves them pointing to mbufs still owned by the mempool. A later
allocation can hand these mbufs to the application, after which CQE
decompression may corrupt their metadata through the stale pointers.
Restore the four fake mbuf pointers on allocation failure in both the
regular and MPRQ vector Rx paths. The producer index does not advance
on failure, so restoring the guards at the original allocation position
is sufficient. This adds work only on the allocation failure path.
Fixes: a2833ecc5ea4 ("mempool: fix get objects from mempool with cache")
Cc: [email protected]
Signed-off-by: Mikhail Mamatov <[email protected]>
Signed-off-by: Pavel Ivashchenko <[email protected]>
---
.mailmap | 1 +
drivers/net/mlx5/mlx5_rxtx_vec.c | 12 ++++++++++++
2 files changed, 13 insertions(+)
diff --git a/.mailmap b/.mailmap
index 2f089326ff..3b10eff3fc 100644
--- a/.mailmap
+++ b/.mailmap
@@ -1089,6 +1089,7 @@ Mike Pattrick <[email protected]>
Mike Sowka <[email protected]>
Mike Stolarchuk <[email protected]>
Mike Ximing Chen <[email protected]>
+Mikhail Mamatov <[email protected]>
Mikolaj Filar <[email protected]>
Milena Olech <[email protected]>
Min Cao <[email protected]>
diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.c b/drivers/net/mlx5/mlx5_rxtx_vec.c
index 1b701801c5..1684f760d4 100644
--- a/drivers/net/mlx5/mlx5_rxtx_vec.c
+++ b/drivers/net/mlx5/mlx5_rxtx_vec.c
@@ -105,6 +105,15 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data *rxq)
n = RTE_MIN(n - MLX5_VPMD_DESCS_PER_LOOP, q_n - elts_idx);
if (rte_mempool_get_bulk(rxq->mp, (void *)elts, n) < 0) {
rxq->stats.rx_nombuf += n;
+ /*
+ * A failed bulk get may have partially filled elts[]
+ * from the cache without dequeueing those objects.
+ * Restore the fake_mbuf guards so a following
+ * decompression does not write into mbufs that are
+ * still owned by the mempool (or by the application).
+ */
+ for (i = 0; i < MLX5_VPMD_DESCS_PER_LOOP; ++i)
+ elts[i] = &rxq->fake_mbuf;
return;
}
if (unlikely(mlx5_mr_btree_len(&rxq->mr_ctrl.cache_bh) > 1)) {
@@ -169,6 +178,9 @@ mlx5_rx_mprq_replenish_bulk_mbuf(struct mlx5_rxq_data *rxq)
n = RTE_MIN(n, rxq->rq_repl_thresh);
if (rte_mempool_get_bulk(rxq->mp, (void *)elts, n) < 0) {
rxq->stats.rx_nombuf += n;
+ /* See mlx5_rx_replenish_bulk_mbuf(). */
+ for (i = 0; i < MLX5_VPMD_DESCS_PER_LOOP; ++i)
+ elts[i] = &rxq->fake_mbuf;
return;
}
rxq->elts_ci += n;
--
2.50.1 (Apple Git-155)
