Hi, From: Dariusz Sosnowski <dsosnow...@nvidia.com> Sent: Tuesday, October 22, 2024 3:06 PM To: Slava Ovsiienko; Bing Zhao; Ori Kam; Suanming Mou; Matan Azrad Cc: dev@dpdk.org Subject: [PATCH v2 00/10] net/mlx5: improve MAC address and VLAN add latency
Whenever a new MAC address is added to the port, mlx5 PMD will: - Add this address to `dev->data->mac_addrs[]`. - Destroy all control flow rules. - Recreate all control flow rules. Similar logic is also implemented for VLAN filters. Because of such logic, the latency of adding the new MAC address (i.e., latency of `rte_eth_dev_mac_addr_add()` function call) is actually linear to number of MAC addresses already configured. Since each operation of creating/destroying a control flow rule, involves an `ioctl()` syscall, on some setups the latency of adding a single MAC address can reach ~100ms, when port is operating with >= 100 MAC addresses. The same problem exists for VLAN filters (and even compounded by it). This patchset aims to resolve these issues, by reworking how mlx5 PMD handles adding/removing MAC addresses and VLAN filters. Instead of recreating all control flow rules, only necessary flow rules will be created/removed on each operation, thus minimizing number of syscalls triggered. Summary of patches: - Patch 1-2 - Extends existing `mlx5_hw_ctrl_flow_type` enum with special variants, which will be used for tracking MAC and VLAN control flow rules. - Patch 3-4 - Refactors HWS code for control flow rule creation to allow creation of specific control flow rules with unicast MAC/VLAN match. Also functions are added for deletion of specific rules. - Patch 5-6 - Prepares the control flow rules list, used by HWS flow engine, to be used by other flow engine. Goal is to reuse the similar logic in Verbs and DV flow engines. - Patch 7-8 - Adjusts legacy flow engines, so that unicast DMAC/VLAN control flow rules are added to the control flow rules list. Also exposes functions for creating/destroying specific ones. - Patch 9-10 - Extends `mlx5_traffic_*` interface with `mlx5_traffic_mac_add/remove` and `mlx5_traffic_vlan_add/remove` functions. They are used in implementations of DPDK APIs for adding/removing MAC addresses/VLAN filters and their goal is to update the set of control flow rules in a minimal number of steps possible, without recreating the rules. As a result of these patches the time to add 128th MAC address, after 127th was added drops **from ~72 ms to ~197 us** (at least on my setup). v2: - Rebase on top of 24.11-rc1. - Added Acked-by tags from v1. Dariusz Sosnowski (10): net/mlx5: track unicast DMAC control flow rules net/mlx5: add checking if unicast flow rule exists net/mlx5: rework creation of unicast flow rules net/mlx5: support destroying unicast flow rules net/mlx5: rename control flow rules types net/mlx5: shared init of control flow rules net/mlx5: add legacy unicast flow rules management net/mlx5: add legacy unicast flow rule registration net/mlx5: add dynamic unicast flow rule management net/mlx5: optimize MAC address and VLAN filter handling drivers/net/mlx5/linux/mlx5_os.c | 3 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.h | 62 +++-- drivers/net/mlx5/mlx5_flow.c | 149 ++++++++++- drivers/net/mlx5/mlx5_flow.h | 36 +++ drivers/net/mlx5/mlx5_flow_hw.c | 349 ++++++++++++++++++++------ drivers/net/mlx5/mlx5_flow_hw_stubs.c | 68 +++++ drivers/net/mlx5/mlx5_mac.c | 41 ++- drivers/net/mlx5/mlx5_trigger.c | 262 ++++++++++++++++++- drivers/net/mlx5/mlx5_vlan.c | 9 +- drivers/net/mlx5/windows/mlx5_os.c | 3 + 11 files changed, 867 insertions(+), 116 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_hw_stubs.c -- 2.39.5 Series applied to next-net-mlx, Kindest regards, Raslan Darawsheh
