On 7/22/2024 11:02 AM, Bruce Richardson wrote:
> When allocating memory for an ethdev, the rte_malloc_socket call used
> only allocates memory on the NUMA node/socket local to the device. This
> means that even if the user wanted to, they could never use a remote NIC
> without also having memory on that NIC's socket.
>
> For example, if we change examples/skeleton/basicfwd.c to have
> SOCKET_ID_ANY as the socket_id parameter for Rx and Tx rings, we should
> be able to run the app cross-numa e.g. as below, where the two PCI
> devices are on socket 1, and core 1 is on socket 0:
>
> ./build/examples/dpdk-skeleton -l 1 --legacy-mem --socket-mem=1024,0 \
> -a a8:00.0 -a b8:00.0
>
> This fails however, with the error:
>
> ETHDEV: failed to allocate private data
> PCI_BUS: Requested device 0000:a8:00.0 cannot be used
>
> We can remove this restriction by doing a fallback call to general
> rte_malloc after a call to rte_malloc_socket fails. This should be safe
> to do because the later ethdev calls to setup Rx/Tx queues all take a
> socket_id parameter, which can be used by applications to enforce the
> requirement for local-only memory for a device, if so desired. [If
> device-local memory is present it will be used as before, while if not
> present the rte_eth_dev_configure call will now pass, but the subsequent
> queue setup calls requesting local memory will fail].
>
> Fixes: e489007a411c ("ethdev: add generic create/destroy ethdev APIs")
> Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richard...@intel.com>
> Signed-off-by: Padraig Connolly <padraig.j.conno...@intel.com>
>
Reviewed-by: Ferruh Yigit <ferruh.yi...@amd.com>
Applied to dpdk-next-net/main, thanks.
