I wanted to revisit how resource overload is handled for hardware offload of FIB entries and rules. At the moment, the in-kernel fib notifier can tell a driver about a route or rule add, replace, and delete, but the notifier can not affect the action. Specifically, in the case of mlxsw if a route or rule add is going to overflow the ASIC resources the only recourse is to abort hardware offload. Aborting offload is akin to taking down the switch as the path from data plane to the control plane simply can not support the traffic bandwidth of the front panel ports. Further, the current state of FIB notifiers is inconsistent with other resources where a driver can affect a user request - e.g., enslavement of a port into a bridge or a VRF.
As a result of the work done over the past 3+ years, I believe we are at a point where we can bring consistency to the stack and offloads, and reliably allow the FIB notifiers to fail a request, pushing an error along with a suitable error message back to the user. Rather than aborting offload when the switch is out of resources, userspace is simply prevented from adding more routes and has a clear indication of why. Patch 1 fixes call_fib_notifiers to extract the errno from the encoded response from handlers. Patches 2-5 allow the call to call_fib_notifiers to fail the add or replace of a route or rule. Patch 6 exports some devlink symbols. Patch 7 adds a simple resource controller to netdevsim to illustrate how a FIB resource controller can limit the number of route entries. David Ahern (7): net: Fix fib notifer to return errno net: Move call_fib_rule_notifiers up in fib_nl_newrule net/ipv4: Move call_fib_entry_notifiers up for new routes net/ipv4: Allow notifier to fail route repolace net/ipv6: Move call_fib6_entry_notifiers up for route adds devlink: Export methods to get and set namespace netdevsim: Add simple FIB resource controller via devlink drivers/net/netdevsim/Makefile | 4 + drivers/net/netdevsim/devlink.c | 281 ++++++++++++++++++++++++++++++++++++++ drivers/net/netdevsim/fib.c | 264 +++++++++++++++++++++++++++++++++++ drivers/net/netdevsim/netdev.c | 12 +- drivers/net/netdevsim/netdevsim.h | 42 ++++++ include/net/devlink.h | 2 + net/core/devlink.c | 6 +- net/core/fib_notifier.c | 5 +- net/core/fib_rules.c | 6 +- net/ipv4/fib_trie.c | 27 +++- net/ipv6/ip6_fib.c | 16 ++- 11 files changed, 652 insertions(+), 13 deletions(-) create mode 100644 drivers/net/netdevsim/devlink.c create mode 100644 drivers/net/netdevsim/fib.c -- 2.11.0
