On 2021-09-01 07:41, Kaladi, Ashok K wrote:
Dear dpdk-dev team,
We would like to propose the following RFC for your review.
A user space application may need access to the packets handled by eventdev
based DPDK application. This application doesn't use mbuf or eventdev based
DPDK APIs. Presently this is not possible without passing packets through
DPDK KNI.
How about using memif and the eventdev RX adapter to solve this problem?
This RFC introduces control packet event adapter (CP adapter) and FIFO library
which can be used by user space applications to access the packets
handled by eventdev based DPDK application. Linux shared memory is used to
keep mempool (mbufs), packets and FIFOs. This allows copy-free access of
packets to both the applications. The Rx and Tx FIFOs are used to exchange
packets between the applications.
On a control-type interface zero-copy or not shouldn't really matter,
and also seem likely to result in higher coupling and a more error-prone
interface between the DPDK app and the other app.
'------------------'
| mempool |
'-------------' '-----------' |..................| '----------'
'-----------'
| | | | | <--- Tx FIFOs | | | |
|
| User space | | | | <--- Alloc FIFO | | Control | |
DPDK |
| App <----> FIFO Lib <----> ---> Rx FIFOs <----> Packet <---->
Eventdev |
| | | | | ---> Free FIFO | | Adapter | |
App |
| | | | | | | | |
|
'-------------' '-----------' | | '----------'
'-----------'
'------------------'
CP adapter does the mbuf allocation and free for user space application. Mbufs
allocated for user space application are put into alloc queue by CP adapter.
The used mbufs are put into free queue by user space application.
The FIFO lib translates packets between rte_mbuf format and a simplified
format which will be used by the user application. FIFO lib doesn't use any
other
DPDK features than rte_mbuf structure. The simplified packet format contains
only
essential parameters like address of the payload, packet length, etc.
- user space app send path: When user space application wants to send
a packet, it takes an mbufs from Alloc FIFO, writes packet to it and puts it
into Rx queue. CP adapter which polls this queue gets the packet and enqueues
to eventdev. Then the mbuf is freed by CP adapter.
- use space app receive path: CP adapter puts the packets to Tx FIFO,
user space application polls the FIFO and gets the packet. After consuming
the packet it puts the mbuf to be freed to free FIFO. CP adapter regularly
reads the free FIFO and frees the mbufs in it.
The CP adapter can service multiple devices and queues. Each queue is
configured with a servicing weight to control the relative frequency with which
the adapter polls the queue, and the event fields to use when constructing
packet events.
API Summary
FIFO APIs:
- FIFO device open/close.
- API to allocate PDU buffer from alloc queue.
- API to send a burst of packets to an Rx FIFO identified by FIFO id and
device id.
- API to receive a burst of packets from Tx FIFO identified by FIFO id and
device id.
- API to put the used buffer to Free queue. Use application will call this
after
using the payload.
Control Packet Adapter APIs:
- Initialization API.
- Device addition and removal APIs
- Device configuration APIs.
- FIFO add and remove APIs.
- Event enqueue and dequeue APIs.
We look forward for your feedback on this proposal. Once we have initial
feedback,
patches will be submitted for review.
One thing to consider is if this FIFO service should provide guaranteed
and/or in-order delivery. That in turn probably depends on if you by
"control" mean slow-path traffic (ARP etc), or more RPC/IPC type control
signaling (e.g. an "add this route" or "retrieve these stats" message).
Thanks & Regards
Ashok Kaladi
