Hi,
This interaction with eventdev introduces some overhead. Isn't it easier to
just create an API to query the available credit for a certain event port?
Regards,
Heng
-----Original Message-----
From: Mattias Rönnblom <mattias.ronnb...@ericsson.com>
Sent: Thursday, April 13, 2023 8:54 AM
To: Jerin Jacob Kollanukkaran <jer...@marvell.com>
Cc: timothy.mcdan...@intel.com; Hemant Agrawal <hemant.agra...@nxp.com>; Harry
van Haaren <harry.van.haa...@intel.com>; dev@dpdk.org; Svante Järvstråt
<svante.jarvst...@ericsson.com>; Heng Wang <heng.w...@ericsson.com>; Stefan
Sundkvist <stefan.sundkv...@ericsson.com>; Peter Nilsson
<peter.nils...@ericsson.com>; Maria Lingemark <maria.lingem...@ericsson.com>
Subject: Event device early back-pressure indication
Hi.
Consider this situation:
An application EAL thread receives an eventdev event (or some other stimuli),
which in turn triggers some action. This action results in a number of new
events being prepared, and a number of associated state changes in the
application.
On attempting to enqueue the newly created batch of RTE_EVENT_OP_NEW events, it
turns out the system is very busy, and the event device back pressures (i.e.,
returns a short count in rte_event_enqueue_new_burst()).
The application may now be a in tough spot, in case:
A) The processing was expensive and/or difficult to reverse (e.g., destructive
changes were made to a packet).
B) The application does not have the option to discard the events (and any
related mbufs).
In this situation, it would be very beneficial to the application if the event
device give could give some assurance that a future enqueue operation will
succeed (in its entirety).
From what I understand from today's Eventdev API, there are no good options.
You *may* be able to do some heuristics based on a event device-specific xstat
(to infer the event device load), but that is not even close to "good". You may
also try some application-level buffering, but that assumes that the
packets/state changes are going to be identical, if they are to be sent at a
later time. It would drive complexity in the app.
One seemingly clean way to solve this issue is to allow pre-allocation of
RTE_NEW_OP_NEW credits. The eventdev API doesn't talk about credits, but at
least in the event device implementations I've come across use some kind of
credit system internally.
uint16_t
rte_event_alloc_new_credits(uint8_t dev_id, uint8_t port_id, uint16_t count);
In addition to this function, the application would also need some way to
indicate, at the point of enqueue, that the credits have already been allocated.
I don't see any need for pre-allocating credits for non-RTE_OP_NEW events.
(Some event devices don't even use credits to track such
events.) Back pressure on RTE_OP_FORWARD usually spells disaster, in one form
of the other.
You could use a bit in the rte_event struct for the purpose of signaling if its
credit is pre-allocated. That would allow this change to happen, without any
changes to the enqueue function prototypes.
However, this would require the event device to scan the event array.
I'm not sure I think there is a use case for mixing pre-allocated and
non-pre-allocated events in the same burst.
If this burst-level separation is good enough, one could either change the
existing rte_enqueue_new_burst() or add a new one. Something like:
uint16_t
rte_enqueue_new_burst(uint8_t dev_id, uint8_t port_id,
const struct rte_event ev[],
uint16_t nb_events, uint32_t flags);
#define RTE_EVENT_FLAG_PRE_CREDITS_ALLOCATED (UINT32_C(1) << 0)
A related shortcoming of the current eventdev API is that the
new_event_threshold is tied to a port, which is impractical for applications
which require different threshold for different kinds of events enqueued on the
same port. One can use different ports, but that approach does not scale, since
there may be significant memory and/or event device hardware resources tied to
ports, and thus you cannot allow for a combinatorial explosion of ports.
This issue could be solve by allowing the application to specify the
new_event_threshold, either per burst, or per event.
Per event doesn't make a lot of sense in practice, I think, since mixing events
with different back pressure points will create head-of-line blocking. An early
low-threshold event may prevent higher-indexed high threshold event in the same
enqueue burst from being enqueued. This is the same reason it usually doesn't
make sense to mix RTE_OP_NEW and RTE_OP_FORWARD events in the same burst.
Although the new_event_threshold seems completely orthogonal to the port to me,
it could still serve as the default.
In case you find this a useful feature, it could be added to the credit
allocation function.
uint16_t
rte_event_alloc_new_credits(uint8_t dev_id, uint8_t port_id, uint32_t
new_event_threshold, uint16_t count);
If that is the only change, the user is required to pre-allocated credits to
use a flexible new_event_threshold.
It seems to me that that might be something you can live with. Or, you add new
enqueue_new_burst() variant where a new_event_threshold parameter is added.
It may also be useful to have a way to return credits, in case not all
allocated was actually needed.
void
rte_event_return_new_credits(...);
Thoughts?
Best regards,
Mattias
