23/07/2022 09:24, Morten Brørup пишет:
+CC: i40e maintainers
+CC: mlx5 maintainers
From: Konstantin Ananyev [mailto:konstantin.v.anan...@yandex.ru]
Sent: Saturday, 23 July 2022 00.35
22/07/2022 17:14, Morten Brørup пишет:
From: Huichao Cai [mailto:chcch...@163.com]
Sent: Friday, 22 July 2022 17.59
At 2022-07-22 23:52:28, "Morten Brørup" <m...@smartsharesystems.com>
wrote:
From: Stephen Hemminger [mailto:step...@networkplumber.org]
Sent: Friday, 22 July 2022 16.49
On Fri, 22 Jul 2022 21:01:50 +0800
Huichao Cai <chcch...@163.com> wrote:
Some NIC drivers support MBUF_FAST_FREE(Device supports
optimization
for fast release of mbufs. When set application must guarantee
that
per-queue all mbufs comes from the same mempool and has refcnt =
1)
offload. In order to adapt to this offload function, add this
API.
Add some test data for this API.
Signed-off-by: Huichao Cai <chcch...@163.com>
The code should just be checking that refcnt == 1 directly.
There are cases where sender passes a cloned mbuf. This is
independent
of the fast free optimization.
Similar to what Linux kernel does with skb_cow().
Olivier just confirmed that MBUF_FAST_FREE requires that the mbufs
are direct and non-segmented, although these requirements are not yet
documented.
This means that you should not generate segmented mbufs with this
patch. I don't know what to do instead; probably fail with an
appropriate errno.
When the bnxt driver sends mbuf, it will take the mbuf segments
apart and hang it to the tx_buf_ring, so there is no mbuf segments when
it is released. Does this mean that there can be mbuf segments?
Only if the bnxt driver also resets the segmentation fields (nb_segs
and next) in those mbufs, which I suppose it does, if it supports
MBUF_FAST_FREE with segmented packets.
However, other Ethernet drivers don't do that, so a generic library
function cannot rely on it. These missing requirements for
MBUF_FAST_FREE is a bug, either in the MBUF_FAST_FREE documentation, or
in the drivers where MBUF_FAST_FREE only works correctly with direct
and non-segmented mbufs.
I believe multi-segment packets work ok with MBUF_FAST_FREE
(as long as other requirements are met).
Looking at the i40e and mlx5 drivers, they both seem to call rte_mempool_put_bulk()
without first calling rte_pktmbuf_prefree_seg(). So segmented packets freed with
MBUF_FAST_FREE, will be stored in the mbuf pool without m->nb_segs and m->next
being reset first.
I don't have deep knowledge of these drivers, so maybe I have overlooked
something.
The point of MBUF_FAST_FREE is to bypass a lot of code under certain
conditions. So I believe that these two undocumented requirements should
remain, so the drivers can bypass this code. Otherwise, don't use
MBUF_FAST_FREE.
Actually, after another look, I think you and Olivier are right -
multi-seg packets should not be used together with MBUF_FAST_FREE.
I forgot that mbuf_prefree() is responsible to reset both 'next'
and 'nb_segs' fields of the mbuf.
It might keep working for some simple forwarding app (like l3fwd),
as most PMDs reset these fields at RX path anyway, but that's just a
coincidence we shouldn't rely on.
We probably need to update l3fwd (and other examples) to dis-allow
MBUF_FAST_FREE when TX_OFFLOAD_MULTI_SEGS is selected.
Konstantin
