There are 5 classes of mlx5 drivers (net\eth, RegEx, vDPA, compress and crypto). The various drivers are registered under the common mlx5 driver and are managed by it. The common driver probing calls in a loop to the probe function of each driver registered to it. Each driver creates for itself all the objects required for communication with the hardware and a global MR cache that manages memory mappings. The management of the caches separately by the different drivers is not very efficient. In fact, the same memory is mapped multiple times to the HW when more than 1 class use the device. This feature will move management to the common driver in two phases.
Phase 1: sharing HW objects between drivers on the same device The communication with the hardware - for any MR handle - is conducted by the Protection Domain, so we are motivated to share it between the drivers. However, to create it, we need to give the context of the device, so the context must also be shared between the drivers. At this point, we will share the next trio between the drivers (CTX, PD, pdn) to create an infrastructure that will allow sharing of dependent objects, particularly the global MR cache. The common driver itself will create this trio individually for all drivers before calling their probe function. Then, as a parameter to the probe function, it will give them a pointer to the structure containing the trio. Phase 2: sharing global MR cache between drivers on the same device The common driver will add to the structure containing the trio and the structure that manages the global MR cache and keep a list of such structures for memory management. In each driver, each queue will manage its own local MR cache. If the queue does not find its cache, it will search the global MR cache shared by all. Caching access will be through the pointer that the driver received as a parameter in probing. Michael Baum (21): net/mlx5: fix shared device context creation error flow net/mlx5: fix PCI probing error flow common/mlx5: add context device structure compress/mlx5: use context device structure crypto/mlx5: use context device structure regex/mlx5: use context device structure net/mlx5: improve probe function on Windows net/mlx5: improve probe function on Linux net/mlx5: improve spawn function net/mlx5: use context device structure net/mlx5: move NUMA node field to context device common/mlx5: add ROCE disable in context device creation vdpa/mlx5: use context device structure mlx5: update device sent to probing mlx5: share context device structure between drivers common/mlx5: add HCA attributes to context device structure regex/mlx5: use HCA attributes from context device vdpa/mlx5: use HCA attributes from context device compress/mlx5: use HCA attributes from context device crypto/mlx5: use HCA attributes from context device net/mlx5: use HCA attributes from context device drivers/common/mlx5/linux/mlx5_common_os.c | 268 ++++++++- drivers/common/mlx5/mlx5_common.c | 273 +++++++++- drivers/common/mlx5/mlx5_common.h | 35 +- drivers/common/mlx5/mlx5_common_private.h | 6 - drivers/common/mlx5/version.map | 2 + drivers/common/mlx5/windows/mlx5_common_os.c | 207 ++++++- drivers/compress/mlx5/mlx5_compress.c | 112 +--- drivers/crypto/mlx5/mlx5_crypto.c | 111 +--- drivers/crypto/mlx5/mlx5_crypto.h | 4 +- drivers/crypto/mlx5/mlx5_crypto_dek.c | 5 +- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 8 +- drivers/net/mlx5/linux/mlx5_mp_os.c | 9 +- drivers/net/mlx5/linux/mlx5_os.c | 543 +++++++++---------- drivers/net/mlx5/linux/mlx5_verbs.c | 55 +- drivers/net/mlx5/mlx5.c | 85 ++- drivers/net/mlx5/mlx5.h | 17 +- drivers/net/mlx5/mlx5_devx.c | 35 +- drivers/net/mlx5/mlx5_flow.c | 6 +- drivers/net/mlx5/mlx5_flow_aso.c | 24 +- drivers/net/mlx5/mlx5_flow_dv.c | 51 +- drivers/net/mlx5/mlx5_flow_verbs.c | 4 +- drivers/net/mlx5/mlx5_mr.c | 14 +- drivers/net/mlx5/mlx5_txpp.c | 27 +- drivers/net/mlx5/windows/mlx5_ethdev_os.c | 14 +- drivers/net/mlx5/windows/mlx5_os.c | 285 ++-------- drivers/regex/mlx5/mlx5_regex.c | 74 +-- drivers/regex/mlx5/mlx5_regex.h | 23 +- drivers/regex/mlx5/mlx5_regex_control.c | 12 +- drivers/regex/mlx5/mlx5_regex_fastpath.c | 18 +- drivers/regex/mlx5/mlx5_rxp.c | 64 ++- drivers/vdpa/mlx5/mlx5_vdpa.c | 210 +------ drivers/vdpa/mlx5/mlx5_vdpa.h | 4 +- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 19 +- drivers/vdpa/mlx5/mlx5_vdpa_lm.c | 6 +- drivers/vdpa/mlx5/mlx5_vdpa_mem.c | 13 +- drivers/vdpa/mlx5/mlx5_vdpa_steer.c | 10 +- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 16 +- 37 files changed, 1414 insertions(+), 1255 deletions(-) -- 2.25.1
