On Wed, Oct 2, 2024 at 6:27 PM Bruce Richardson
<bruce.richard...@intel.com> wrote:
>
> On Wed, Oct 02, 2024 at 05:57:08PM +0200, David Marchand wrote:
> > While invoking telemetry commands (which may happen at any time,
> > out of control of the application), an application thread may
> > concurrently add/remove ports.
> > The telemetry callbacks may then access partially
> > initialised/uninitialised ethdev data.
> >
> > Reuse the ethdev lock that protects port allocation/destruction.
> >
> > Fixes: c190daedb9b1 ("ethdev: add telemetry callbacks")
> > Cc: sta...@dpdk.org
> >
> > Signed-off-by: David Marchand <david.march...@redhat.com>
> > ---
> > lib/ethdev/rte_ethdev_telemetry.c | 93 +++++++++++++++++++------------
> > 1 file changed, 56 insertions(+), 37 deletions(-)
> >
> > diff --git a/lib/ethdev/rte_ethdev_telemetry.c
> > b/lib/ethdev/rte_ethdev_telemetry.c
> > index 8031a58595..7f9c924209 100644
> > --- a/lib/ethdev/rte_ethdev_telemetry.c
> > +++ b/lib/ethdev/rte_ethdev_telemetry.c
> > @@ -6,6 +6,7 @@
> > #include <stdlib.h>
> >
> > #include <rte_kvargs.h>
> > +#include <rte_spinlock.h>
> > #include <rte_telemetry.h>
> >
> > #include "rte_ethdev.h"
> > @@ -1403,43 +1404,61 @@ eth_dev_handle_port_tm_node_caps(const char *cmd
> > __rte_unused,
> > return ret;
> > }
> >
> > +#define ETHDEV_TELEMETRY_HANDLERS \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/list", eth_dev_handle_port_list, \
> > + "Returns list of available ethdev ports. Takes no
> > parameters") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/stats", eth_dev_handle_port_stats, \
> > + "Returns the common stats for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/xstats",
> > eth_dev_handle_port_xstats, \
> > + "Returns the extended stats for a port. Parameters: int
> > port_id,hide_zero=true|false(Optional for indicates hide zero xstats)") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/dump_priv",
> > eth_dev_handle_port_dump_priv, \
> > + "Returns dump private information for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/link_status",
> > eth_dev_handle_port_link_status, \
> > + "Returns the link status for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/info", eth_dev_handle_port_info, \
> > + "Returns the device info for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/module_eeprom",
> > eth_dev_handle_port_module_eeprom, \
> > + "Returns module EEPROM info with SFF specs. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/macs", eth_dev_handle_port_macs, \
> > + "Returns the MAC addresses for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/flow_ctrl",
> > eth_dev_handle_port_flow_ctrl, \
> > + "Returns flow ctrl info for a port. Parameters: int port_id")
> > \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/rx_queue", eth_dev_handle_port_rxq,
> > \
> > + "Returns Rx queue info for a port. Parameters: int port_id,
> > int queue_id (Optional if only one queue)") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/tx_queue", eth_dev_handle_port_txq,
> > \
> > + "Returns Tx queue info for a port. Parameters: int port_id,
> > int queue_id (Optional if only one queue)") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/dcb", eth_dev_handle_port_dcb, \
> > + "Returns DCB info for a port. Parameters: int port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/rss_info",
> > eth_dev_handle_port_rss_info, \
> > + "Returns RSS info for a port. Parameters: int port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/fec", eth_dev_handle_port_fec, \
> > + "Returns FEC info for a port. Parameters: int port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/vlan", eth_dev_handle_port_vlan, \
> > + "Returns VLAN info for a port. Parameters: int port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/tm_capability",
> > eth_dev_handle_port_tm_caps, \
> > + "Returns TM Capabilities info for a port. Parameters: int
> > port_id") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/tm_level_capability",
> > eth_dev_handle_port_tm_level_caps, \
> > + "Returns TM Level Capabilities info for a port. Parameters:
> > int port_id, int level_id (see tm_capability for the max)") \
> > + ETHDEV_TELEMETRY_HANDLER("/ethdev/tm_node_capability",
> > eth_dev_handle_port_tm_node_caps, \
> > + "Returns TM Node Capabilities info for a port. Parameters:
> > int port_id, int node_id (see tm_capability for the max)")
> > +
> > +#define ETHDEV_TELEMETRY_HANDLER(command, func, usage) \
> > +static int func ## _locked(const char *cmd __rte_unused, const char
> > *params, \
> > + struct rte_tel_data *d) \
> > +{ \
> > + int ret; \
> > + rte_spinlock_lock(rte_mcfg_ethdev_get_lock()); \
> > + ret = func(cmd, params, d); \
> > + rte_spinlock_unlock(rte_mcfg_ethdev_get_lock()); \
> > + return ret; \
> > +}
> > +ETHDEV_TELEMETRY_HANDLERS
> > +#undef ETHDEV_TELEMETRY_HANDLER
> > +
>
> Not really a massive fan of such use of macros in the code, since I think
> it makes things obscure for the casual reader. However, I see why this
> approach has been taken. I think the macro code needs some documentation
> explaining why this was done this way.
I can add comments explaining how this is for protecting accesses to port.
>
> > RTE_INIT(ethdev_init_telemetry)
> > {
> > - rte_telemetry_register_cmd("/ethdev/list", eth_dev_handle_port_list,
> > - "Returns list of available ethdev ports. Takes no
> > parameters");
> > - rte_telemetry_register_cmd("/ethdev/stats", eth_dev_handle_port_stats,
> > - "Returns the common stats for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/xstats",
> > eth_dev_handle_port_xstats,
> > - "Returns the extended stats for a port. Parameters:
> > int port_id,hide_zero=true|false(Optional for indicates hide zero xstats)");
> > - rte_telemetry_register_cmd("/ethdev/dump_priv",
> > eth_dev_handle_port_dump_priv,
> > - "Returns dump private information for a port.
> > Parameters: int port_id");
> > - rte_telemetry_register_cmd("/ethdev/link_status",
> > - eth_dev_handle_port_link_status,
> > - "Returns the link status for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/info", eth_dev_handle_port_info,
> > - "Returns the device info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/module_eeprom",
> > eth_dev_handle_port_module_eeprom,
> > - "Returns module EEPROM info with SFF specs.
> > Parameters: int port_id");
> > - rte_telemetry_register_cmd("/ethdev/macs", eth_dev_handle_port_macs,
> > - "Returns the MAC addresses for a port. Parameters:
> > int port_id");
> > - rte_telemetry_register_cmd("/ethdev/flow_ctrl",
> > eth_dev_handle_port_flow_ctrl,
> > - "Returns flow ctrl info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/rx_queue",
> > eth_dev_handle_port_rxq,
> > - "Returns Rx queue info for a port. Parameters: int
> > port_id, int queue_id (Optional if only one queue)");
> > - rte_telemetry_register_cmd("/ethdev/tx_queue",
> > eth_dev_handle_port_txq,
> > - "Returns Tx queue info for a port. Parameters: int
> > port_id, int queue_id (Optional if only one queue)");
> > - rte_telemetry_register_cmd("/ethdev/dcb", eth_dev_handle_port_dcb,
> > - "Returns DCB info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/rss_info",
> > eth_dev_handle_port_rss_info,
> > - "Returns RSS info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/fec", eth_dev_handle_port_fec,
> > - "Returns FEC info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/vlan", eth_dev_handle_port_vlan,
> > - "Returns VLAN info for a port. Parameters: int
> > port_id");
> > - rte_telemetry_register_cmd("/ethdev/tm_capability",
> > eth_dev_handle_port_tm_caps,
> > - "Returns TM Capabilities info for a port. Parameters:
> > int port_id");
> > - rte_telemetry_register_cmd("/ethdev/tm_level_capability",
> > eth_dev_handle_port_tm_level_caps,
> > - "Returns TM Level Capabilities info for a port.
> > Parameters: int port_id, int level_id (see tm_capability for the max)");
> > - rte_telemetry_register_cmd("/ethdev/tm_node_capability",
> > eth_dev_handle_port_tm_node_caps,
> > - "Returns TM Node Capabilities info for a port.
> > Parameters: int port_id, int node_id (see tm_capability for the max)");
> > +#define ETHDEV_TELEMETRY_HANDLER(command, func, usage) \
> > + rte_telemetry_register_cmd(command, func ## _locked, usage);
> > + ETHDEV_TELEMETRY_HANDLERS
> > +#undef ETHDEV_TELEMETRY_HANDLER
> > }
>
> An alternative to this macro-fu, is to just define a single ethdev
> telemetry function, and within that, take the lock and then dispatch to the
> appropriate subfunction based upon the actual command coming in. The
> dispatch may be slightly slower due to the additional text matching (only
> from byte 8 onwards, so very short strings), but I think the code could be
> a simpler in C rather than in macros, and the perf impact for telemetry is
> likely to be negligible, compared to the overhead of the socket I/O etc.
Hopefully, dispatching performance is not important here.
(skipping the byte 8 stuff) is your proposal something like:
static int
one_callback_to_rule_them_all(const char *cmd, const char *params,
struct rte_tel_data *d)
{
telemetry_cb cb = NULL;
int ret = -EINVAL;
if (strcmp(cmd, "/ethdev/list") == 0) { cb = eth_dev_handle_port_list; }
else if (strcmp(cmd, "/ethdev/stats") == 0) { cb =
eth_dev_handle_port_stats; }
else if (strcmp(cmd, "/ethdev/xstats") == 0) { cb =
eth_dev_handle_port_xstats; }
...
if (cb != NULL) {
rte_spinlock_lock(rte_mcfg_ethdev_get_lock());
ret = cb(cmd, params, d);
rte_spinlock_unlock(rte_mcfg_ethdev_get_lock());
}
return ret;
}
RTE_INIT(ethdev_init_telemetry)
{
rte_telemetry_register_cmd("/ethdev/list", one_callback_to_rule_them_all,
"Returns list of available ethdev ports. Takes
no parameters");
rte_telemetry_register_cmd("/ethdev/stats",
one_callback_to_rule_them_all,
"Returns the common stats for a port.
Parameters: int port_id");
rte_telemetry_register_cmd("/ethdev/xstats",
one_callback_to_rule_them_all,
"Returns the extended stats for a port.
Parameters: int port_id,hide_zero=true|false(Optional for indicates
hide zero xstats)");
...
Which I find inelegant and not that great for maintenance: having the
same info (especially strings that are only evaluated at runtime) in
two locations is a call to inadvertence bugs.
The macros I propose are a way to avoid splitting the callback
function and the command names.
Then addition of a handler is tied with a single declaration.
+ ETHDEV_TELEMETRY_HANDLER("/ethdev/newstuff", eth_dev_new_handler, \
+ "New shiny command. Takes no parameters") \
--
David Marchand
