> >> > >> On 2/4/2022 12:54 PM, Ciara Loftus wrote: > >>> Secondary process support had been disabled for the AF_XDP PMD > >>> because there was no logic in place to share the AF_XDP socket > >>> file descriptors between the processes. This commit introduces > >>> this logic using the IPC APIs. > >>> > >>> Since AF_XDP rings are single-producer single-consumer, rx/tx > >>> in the secondary process is disabled. However other operations > >>> including retrieval of stats are permitted. > >>> > >>> Signed-off-by: Ciara Loftus <ciara.lof...@intel.com> > >>> > >>> --- > >>> v1 -> v2: > >>> * Rebase to next-net > >>> > >>> RFC -> v1: > >>> * Added newline to af_xdp.rst > >>> * Fixed spelling errors > >>> * Fixed potential NULL dereference in init_internals > >>> * Fixed potential free of address-of expression in > afxdp_mp_request_fds > >>> --- > >>> doc/guides/nics/af_xdp.rst | 9 ++ > >>> doc/guides/nics/features/af_xdp.ini | 1 + > >>> doc/guides/rel_notes/release_22_03.rst | 1 + > >>> drivers/net/af_xdp/rte_eth_af_xdp.c | 210 > >> +++++++++++++++++++++++-- > >>> 4 files changed, 207 insertions(+), 14 deletions(-) > >>> > >>> diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst > >>> index db02ea1984..eb4eab28a8 100644 > >>> --- a/doc/guides/nics/af_xdp.rst > >>> +++ b/doc/guides/nics/af_xdp.rst > >>> @@ -141,4 +141,13 @@ Limitations > >>> NAPI context from a watchdog timer instead of from softirqs. More > >> information > >>> on this feature can be found at [1]. > >>> > >>> +- **Secondary Processes** > >>> + > >>> + Rx and Tx are not supported for secondary processes due to the > single- > >> producer > >>> + single-consumer nature of the AF_XDP rings. However other > operations > >> including > >>> + statistics retrieval are permitted. > >> > >> Hi Ciara, > >> > >> Isn't this limitation same for all PMDs, like not both primary & secondary > can > >> Rx/Tx > >> from same queue at the same time. > >> But primary can initiallize the PMD and secondary can do the datapath, > >> or isn't af_xdp supports multiple queue, if so some queues can be used > by > >> primary and some by secondary for datapath. > >> > >> Is there anyhing special for af_xdp that prevents it? > > > > Hi Ferruh, > > > > Thanks for the review. > > Each queue of the PMD corresponds to a new AF_XDP socket. > > Each socket has an RX and TX ring that is mmapped from the kernel to > userspace and this mapping is only valid for the primary process. > > I did not figure out a way to share that mapping with the secondary process > successfully. Can you think of anything that might work? > > > > Does the application knows the buffer address for the Rx/Tx, or is > abstracted to the 'fd'?
The application knows the buffer address of the Rx/Tx rings. We pass a pointer to these rings to the libbpf xsk_socket__create API, which sets up the mappings: http://code.dpdk.org/dpdk/v21.11/source/drivers/net/af_xdp/rte_eth_af_xdp.c#L1291 Then later on in the datapath we operate directly on those rings: http://code.dpdk.org/dpdk/v21.11/source/drivers/net/af_xdp/rte_eth_af_xdp.c#L268 The fd is used in the datapath, but just for the syscalls (recvfrom/poll/send). > If only 'fd' is used, this patch already converts 'fd' between > processes. > cc'ed Anatoly, but what I understand is after MP fd conversion: > Primary process: FD=x > Secondary process: FD=y > And both x & y points to exact same socket in the kernel side. > > At least this is how it works for the 'tap' interface, and that is > why 'fs' are in the process_private area and converted between primary > and secondary, I thought it will be same for the xdp socket. > > Did you test the secondary Rx/Tx in the secondary after this patch?
