Hi, I've reviewed the series, and it looks good to me. I see we did not introduce new indirect referencing on the datapath (just replaced rte_eth_devices[] being hidden with the new rte_eth_fp_ops[].) My only concern - we'll get two places where pointers to the PMDs routines are stored, and it means potential unsynchro between them, but I do not see the actual scenario for that. For example, mlx5 PMD proposes multiple tx/rx_burst routines, and actual routine selection happens on dev_start(), but then it is not supposed to be changed till dev_stop(). The internal PMD checks like this: "if (dev->rx_pkt_burst == mlx5_rx_burst)" supposed to continue working OK as well.
Hence, for series: Acked-by: Viacheslav Ovsiienko <viachesl...@nvidia.com> With best regards, Slava > -----Original Message----- > From: Konstantin Ananyev <konstantin.anan...@intel.com> > Sent: Thursday, October 7, 2021 14:28 > To: dev@dpdk.org > Cc: xiaoyun...@intel.com; ano...@marvell.com; jer...@marvell.com; > ndabilpu...@marvell.com; adwiv...@marvell.com; > shepard.sie...@atomicrules.com; ed.cz...@atomicrules.com; > john.mil...@atomicrules.com; irussk...@marvell.com; > ajit.khapa...@broadcom.com; somnath.ko...@broadcom.com; > rahul.lakkire...@chelsio.com; hemant.agra...@nxp.com; > sachin.sax...@oss.nxp.com; haiyue.w...@intel.com; johnd...@cisco.com; > hyon...@cisco.com; qi.z.zh...@intel.com; xiao.w.w...@intel.com; > humi...@huawei.com; yisen.zhu...@huawei.com; ouli...@huawei.com; > beilei.x...@intel.com; jingjing...@intel.com; qiming.y...@intel.com; Matan > Azrad <ma...@nvidia.com>; Slava Ovsiienko <viachesl...@nvidia.com>; > sthem...@microsoft.com; NBU-Contact-longli <lon...@microsoft.com>; > heinrich.k...@corigine.com; kirankum...@marvell.com; > andrew.rybche...@oktetlabs.ru; mcze...@marvell.com; > jiawe...@trustnetic.com; jianw...@trustnetic.com; > maxime.coque...@redhat.com; chenbo....@intel.com; NBU-Contact-Thomas > Monjalon <tho...@monjalon.net>; ferruh.yi...@intel.com; m...@ashroe.eu; > jay.jayatheert...@intel.com; Konstantin Ananyev > <konstantin.anan...@intel.com> > Subject: [PATCH v5 0/7] hide eth dev related structures > > v5 changes: > - Fix spelling (Thomas/David) > - Rename internal helper functions (David) > - Reorder patches and update commit messages (Thomas) > - Update comments (Thomas) > - Changed layout in rte_eth_fp_ops, to group functions and > related data based on their functionality: > first 64B line for Rx, second one for Tx. > Didn't observe any real performance difference comparing to > original layout. Though decided to keep a new one, as it seems > a bit more plausible. > > v4 changes: > - Fix secondary process attach (Pavan) > - Fix build failure (Ferruh) > - Update lib/ethdev/verion.map (Ferruh) > Note that moving newly added symbols from EXPERIMENTAL to DPDK_22 > section makes checkpatch.sh to complain. > > v3 changes: > - Changes in public struct naming (Jerin/Haiyue) > - Split patches > - Update docs > - Shamelessly included Andrew's patch: > https://patches.dpdk.org/project/dpdk/patch/20210928154856.1015020- > 1-andrew.rybche...@oktetlabs.ru/ > into these series. > I have to do similar thing here, so decided to avoid duplicated effort. > > The aim of these patch series is to make rte_ethdev core data structures > (rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback, etc.) internal to DPDK > and not visible to the user. > That should allow future possible changes to core ethdev related structures to > be transparent to the user and help to improve ABI/API stability. > Note that current ethdev API is preserved, but it is a formal ABI break. > > The work is based on previous discussions at: > https://www.mail-archive.com/dev@dpdk.org/msg211405.html > https://www.mail-archive.com/dev@dpdk.org/msg216685.html > and consists of the following main points: > 1. Copy public 'fast' function pointers (rx_pkt_burst(), etc.) and > related data pointer from rte_eth_dev into a separate flat array. > We keep it public to still be able to use inline functions for these > 'fast' calls (like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown. > Note that apart from function pointers itself, each element of this > flat array also contains two opaque pointers for each ethdev: > 1) a pointer to an array of internal queue data pointers > 2) points to array of queue callback data pointers. > Note that exposing this extra information allows us to avoid extra > changes inside PMD level, plus should help to avoid possible > performance degradation. > 2. Change implementation of 'fast' inline ethdev functions > (rte_eth_rx_burst(), etc.) to use new public flat array. > While it is an ABI breakage, this change is intended to be transparent > for both users (no changes in user app is required) and PMD developers > (no changes in PMD is required). > One extra note - with new implementation RX/TX callback invocation > will cost one extra function call with this changes. That might cause > some slowdown for code-path with RX/TX callbacks heavily involved. > Hope such trade-off is acceptable for the community. > 3. Move rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback and related > things into internal header: <ethdev_driver.h>. > > That approach was selected to: > - Avoid(/minimize) possible performance losses. > - Minimize required changes inside PMDs. > > Performance testing results (ICX 2.0GHz, E810 (ice)): > - testpmd macswap fwd mode, plus > a) no RX/TX callbacks: > no actual slowdown observed > b) bpf-load rx 0 0 JM ./dpdk.org/examples/bpf/t3.o: > ~2% slowdown > - l3fwd: no actual slowdown observed > > Would like to thank everyone who already reviewed and tested previous > versions of these series. All other interested parties please don't be shy and > provide your feedback. > > Andrew Rybchenko (1): > ethdev: remove legacy Rx descriptor done API > > Konstantin Ananyev (6): > ethdev: allocate max space for internal queue array > ethdev: change input parameters for rx_queue_count > ethdev: copy fast-path API into separate structure > ethdev: make fast-path functions to use new flat array > ethdev: add API to retrieve multiple ethernet addresses > ethdev: hide eth dev related structures > > app/test-pmd/config.c | 23 +- > doc/guides/nics/features.rst | 6 +- > doc/guides/rel_notes/deprecation.rst | 5 - > doc/guides/rel_notes/release_21_11.rst | 21 ++ > drivers/common/octeontx2/otx2_sec_idev.c | 2 +- > drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 2 +- > drivers/net/ark/ark_ethdev_rx.c | 4 +- > drivers/net/ark/ark_ethdev_rx.h | 3 +- > drivers/net/atlantic/atl_ethdev.h | 2 +- > drivers/net/atlantic/atl_rxtx.c | 9 +- > drivers/net/bnxt/bnxt_ethdev.c | 8 +- > drivers/net/cxgbe/base/adapter.h | 2 +- > drivers/net/dpaa/dpaa_ethdev.c | 9 +- > drivers/net/dpaa2/dpaa2_ethdev.c | 9 +- > drivers/net/dpaa2/dpaa2_ptp.c | 2 +- > drivers/net/e1000/e1000_ethdev.h | 10 +- > drivers/net/e1000/em_ethdev.c | 1 - > drivers/net/e1000/em_rxtx.c | 21 +- > drivers/net/e1000/igb_ethdev.c | 2 - > drivers/net/e1000/igb_rxtx.c | 21 +- > drivers/net/enic/enic_ethdev.c | 12 +- > drivers/net/fm10k/fm10k.h | 5 +- > drivers/net/fm10k/fm10k_ethdev.c | 1 - > drivers/net/fm10k/fm10k_rxtx.c | 29 +- > drivers/net/hns3/hns3_rxtx.c | 7 +- > drivers/net/hns3/hns3_rxtx.h | 2 +- > drivers/net/i40e/i40e_ethdev.c | 1 - > drivers/net/i40e/i40e_rxtx.c | 30 +- > drivers/net/i40e/i40e_rxtx.h | 4 +- > drivers/net/iavf/iavf_rxtx.c | 4 +- > drivers/net/iavf/iavf_rxtx.h | 2 +- > drivers/net/ice/ice_rxtx.c | 4 +- > drivers/net/ice/ice_rxtx.h | 2 +- > drivers/net/igc/igc_ethdev.c | 1 - > drivers/net/igc/igc_txrx.c | 23 +- > drivers/net/igc/igc_txrx.h | 5 +- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 - > drivers/net/ixgbe/ixgbe_ethdev.h | 5 +- > drivers/net/ixgbe/ixgbe_rxtx.c | 22 +- > drivers/net/mlx5/mlx5_rx.c | 26 +- > drivers/net/mlx5/mlx5_rx.h | 2 +- > drivers/net/netvsc/hn_rxtx.c | 4 +- > drivers/net/netvsc/hn_var.h | 3 +- > drivers/net/nfp/nfp_rxtx.c | 4 +- > drivers/net/nfp/nfp_rxtx.h | 3 +- > drivers/net/octeontx2/otx2_ethdev.c | 1 - > drivers/net/octeontx2/otx2_ethdev.h | 3 +- > drivers/net/octeontx2/otx2_ethdev_ops.c | 20 +- > drivers/net/sfc/sfc_ethdev.c | 29 +- > drivers/net/thunderx/nicvf_ethdev.c | 3 +- > drivers/net/thunderx/nicvf_rxtx.c | 4 +- > drivers/net/thunderx/nicvf_rxtx.h | 2 +- > drivers/net/txgbe/txgbe_ethdev.h | 3 +- > drivers/net/txgbe/txgbe_rxtx.c | 4 +- > drivers/net/vhost/rte_eth_vhost.c | 4 +- > drivers/net/virtio/virtio_ethdev.c | 1 - > lib/ethdev/ethdev_driver.h | 148 +++++++++ > lib/ethdev/ethdev_private.c | 83 +++++ > lib/ethdev/ethdev_private.h | 7 + > lib/ethdev/rte_ethdev.c | 89 ++++-- > lib/ethdev/rte_ethdev.h | 288 ++++++++++++------ > lib/ethdev/rte_ethdev_core.h | 171 +++-------- > lib/ethdev/version.map | 8 +- > lib/eventdev/rte_event_eth_rx_adapter.c | 2 +- > lib/eventdev/rte_event_eth_tx_adapter.c | 2 +- > lib/eventdev/rte_eventdev.c | 2 +- > lib/metrics/rte_metrics_telemetry.c | 2 +- > 67 files changed, 677 insertions(+), 564 deletions(-) > > -- > 2.26.3
