On 1/18/2023 1:53 AM, Du, Frank wrote:
>
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yi...@amd.com>
>> Sent: Tuesday, January 17, 2023 5:14 PM
>> To: Du, Frank <frank...@intel.com>; Loftus, Ciara <ciara.lof...@intel.com>
>> Cc: dev@dpdk.org
>> Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs
>>
>> On 1/17/2023 1:35 AM, Du, Frank wrote:
>>
>> Moved down, please don't top post.
>>
>>>>
>>>> -----Original Message-----
>>>> From: Ferruh Yigit, <ferruh.yi...@amd.com>
>>>> Sent: Monday, January 16, 2023 9:15 PM
>>>> To: Du, Frank <frank...@intel.com>; Loftus, Ciara
>>>> <ciara.lof...@intel.com>
>>>> Cc: dev@dpdk.org
>>>> Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs
>>>>
>>>> On 12/12/2022 12:48 AM, Frank Du wrote:
>>>>> Get from /sys/class/net/{if}/device/numa_node.
>>>>>
>>>>> Signed-off-by: Frank Du <frank...@intel.com>
>>>>> ---
>>>>> drivers/net/af_xdp/rte_eth_af_xdp.c | 17 ++++++++++++++---
>>>>> 1 file changed, 14 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
>>>>> b/drivers/net/af_xdp/rte_eth_af_xdp.c
>>>>> index b6ec9bf490..38b9d36ab5 100644
>>>>> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
>>>>> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
>>>>> @@ -39,6 +39,7 @@
>>>>> #include <rte_power_intrinsics.h>
>>>>>
>>>>> #include "compat.h"
>>>>> +#include "eal_filesystem.h"
>>>>>
>>>>> #ifndef SO_PREFER_BUSY_POLL
>>>>> #define SO_PREFER_BUSY_POLL 69
>>>>> @@ -2038,9 +2039,6 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device
>> *dev)
>>>>> return -EINVAL;
>>>>> }
>>>>>
>>>>> - if (dev->device.numa_node == SOCKET_ID_ANY)
>>>>> - dev->device.numa_node = rte_socket_id();
>>>>> -
>>>>> if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx,
>>>>> &xsk_queue_cnt, &shared_umem, prog_path,
>>>>> &busy_budget, &force_copy) < 0) { @@ -2053,6
>> +2051,19 @@
>>>>> rte_pmd_af_xdp_probe(struct rte_vdev_device *dev)
>>>>> return -EINVAL;
>>>>> }
>>>>>
>>>>> + /* get numa node id from net sysfs */
>>>>> + if (dev->device.numa_node == SOCKET_ID_ANY) {
>>>>> + unsigned long numa = 0;
>>>>> + char numa_path[PATH_MAX];
>>>>> +
>>>>> + snprintf(numa_path, sizeof(numa_path),
>> "/sys/class/net/%s/device/numa_node",
>>>>> + if_name);
>>>>> + if (eal_parse_sysfs_value(numa_path, &numa) != 0)
>>>>> + dev->device.numa_node = rte_socket_id();
>>>>> + else
>>>>> + dev->device.numa_node = numa;
>>>>> + }
>>>>> +
>>>>> busy_budget = busy_budget == -1 ? ETH_AF_XDP_DFLT_BUSY_BUDGET :
>>>>> busy_budget;
>>>>>
>>>>
>>>> Hi Frank,
>>>>
>>>> It looks reasonable to set virtual DPDK af_xdp device socket to actual
>> underlying device socket. And as I checked quickly, it works as expected.
>>>>
>>>> But what is the impact and motivation of the patch? In other words why you
>> are doing this patch and what output you are expecting as a result?
>>>> Did you able to do any performance testing, and are you observing any
>> difference before and after this test?
>>>>
>>>
>>> Hi ferruh,
>>>
>>> Our application use rte_eth_dev_socket_id to query the socket that a NIC
>>> port
>> connected, then allocate lcore/memory according to this affinity.
>>>
>>> The remote memory access is really slow compared to local.
>>>
>>
>> As you observing any performance gain after change? If so, how much?
>
> Hi Ferruh,
>
> The NIC in our setup is on the socket 1, if our workload(both memory and cpu)
> are running on socket 0, it can get max 12g/s throughput on a single core.
> With this patch, the workload is running on the correct socket 1 cpus, it can
> get up to 16g/s on a single core.
Thanks for clarification.
