Hi Dave, This series, mostly from Vlad, is the first part of ongoing work to improve mlx5 tc flow handling by removing dependency on rtnl_lock and providing a more fine-grained locking and rcu safe data structures.
For more information please see tag log below. Please pull and let me know if there is any problem. Thanks, Saeed. --- The following changes since commit 85fd8011475e86265beff7b7617c493c247f5356: Merge branch 'bnxt_en-TPA-57500' (2019-07-29 14:19:09 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2019-07-29 for you to fetch changes up to b6fac0b46a1a76024698d240f0f9aac552f897b7: net/mlx5e: Protect tc flow table with mutex (2019-07-29 16:40:26 -0700) ---------------------------------------------------------------- mlx5-updates-2019-07-29 This series includes updates to mlx5 driver, 1) Simplifications, cleanup and warning prints improvements 2) From Vlad Buslov: Refactor mlx5 tc flow handling for unlocked execution (Part 1) Currently, all cls API hardware offloads driver callbacks require caller to hold rtnl lock when calling them. Cls API has already been updated to update software filters in parallel (on classifiers that support unlocked execution), however hardware offloads code still obtains rtnl lock before calling driver tc callbacks. This set implements partial support for unlocked execution that is leveraged by follow up refactorings in specific mlx5 tc subsystems and patch to cls API that implements API that allows drivers to register their callbacks as rtnl-unlocked. In mlx5 tc code mlx5e_tc_flow is the main structure that is used to represent tc filter. Currently, code the structure itself and its handlers in both tc and eswitch layers do not implement any kind of synchronizations and assume external global synchronization provided by rtnl lock instead. Implement following changes to remove dependency on rtnl lock in flow handling code that are intended to be used a groundwork for following changes to provide fully rtnl-independent mlx5 tc: - Extend struct mlx5e_tc_flow with atomic reference counter and rcu to allow concurrent access from multiple tc and neigh update workqueue instances without introducing any additional locks specific to the structure. Its 'flags' field type is changed to atomic bitmask ops which is necessary for tc to interact with other concurrent tc instances or concurrent neigh update that need to skip flows that are not fully initialized (new INIT_DONE flow flag) and can change the flags according to neighbor state (flipping OFFLOADED flag). - Protect unready flows list by new uplink_priv->unready_flows_lock mutex. - Convert calls to netdev APIs that require rtnl lock in flow handling code to their rcu counterparts. - Modify eswitch code that is called from tc layer and assume implicit external synchronization to be concurrency safe: change esw->offloads.num_flows type to atomic integer and re-arrange esw->state_lock usage to protect additional data. Some of approaches to synchronizations presented in this patch set are quite complicated (lockless concurrent usage of data structures with rcu and reference counting, using fine-grained locking when necessary, retry mechanisms to handle concurrent insertion of another instance of data structure with same key, etc.). This is necessary to allow calling the firmware in parallel in most cases, which is the main motivation of this change since firmware calls are mach heavier operation than atomic operations, multitude of locks and potential multiple retries during concurrent accesses to same elements. ---------------------------------------------------------------- Eli Britstein (1): net/mlx5e: Simplify get_route_and_out_devs helper function Huy Nguyen (1): net/mlx5e: Print a warning when LRO feature is dropped or not allowed Saeed Mahameed (2): net/mlx5e: Avoid warning print when not required net/mlx5e: Improve ethtool rxnfc callback structure Vlad Buslov (8): net/mlx5e: Extend tc flow struct with reference counter net/mlx5e: Change flow flags type to unsigned long net/mlx5e: Protect tc flows hashtable with rcu net/mlx5e: Protect unready flows with dedicated lock net/mlx5e: Eswitch, change offloads num_flows type to atomic64 net/mlx5e: Eswitch, use state_lock to synchronize vlan change net/mlx5e: Rely on rcu instead of rtnl lock when getting upper dev net/mlx5e: Protect tc flow table with mutex wenxu (1): net/mlx5e: Fix unnecessary flow_block_cb_is_busy call drivers/net/ethernet/mellanox/mlx5/core/en/fs.h | 13 +- .../net/ethernet/mellanox/mlx5/core/en/tc_tun.c | 33 +- .../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 28 +- .../ethernet/mellanox/mlx5/core/en_fs_ethtool.c | 11 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 16 +- drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 12 +- drivers/net/ethernet/mellanox/mlx5/core/en_rep.h | 2 + drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 492 +++++++++++++-------- drivers/net/ethernet/mellanox/mlx5/core/en_tc.h | 26 +- drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 16 +- drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 3 +- .../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 27 +- 12 files changed, 424 insertions(+), 255 deletions(-)