On 6/7/22 00:30, Thomas Monjalon wrote:
It seems we share a common understanding
and we need to agree on a good wording
for the most meaningful API.
Questions inline below:
06/06/2022 19:15, Andrew Rybchenko:
On 6/6/22 16:16, Spike Du wrote:
Hi Andrew,
Please see below for "fill threshold" concept, I'm ok with other
comments about code.
Regards,
Spike.
From: Andrew Rybchenko <andrew.rybche...@oktetlabs.ru>
On 6/3/22 15:48, Spike Du wrote:
Fill threshold describes the fullness of a Rx queue. If the Rx queue
fullness is above the threshold, the device will trigger the event
RTE_ETH_EVENT_RX_FILL_THRESH.
Sorry, I'm not sure that I understand. As far as I know the process to add
more Rx buffers to Rx queue is called 'refill' in many drivers. So fill level
is a
number (or percentage) of free buffers in an Rx queue.
If so, fill threashold should be a minimum fill level and below the level we
should generate an event.
However reading the first paragraph of the descrition it looks like you mean
oposite thing - a number (or percentage) of ready Rx buffers with received
packets.
I think that the term "fill threshold" is suggested by me, but I did it with
mine
understanding of the added feature. Now I'm confused.
Moreover, I don't understand how "fill threshold" could be in terms of ready
Rx buffers. HW simply don't really know when ready Rx buffers are
processed by SW. So, HW can't say for sure how many ready Rx buffers are
pending. It could be calculated as Rx queue size minus number of free Rx
buffers, but it is imprecise. First of all not all Rx descriptors could be used.
Second, HW ring size could differ queue size specified in SW.
Queue size specified in SW could just limit maximum nubmer of free Rx
buffers provided by the driver.
Let me use other terms because "fill"/"refill" is also ambiguous to me.
In a RX ring, there are Rx buffers with received packets, you call it "ready Rx
buffers", there is a RTE api rte_eth_rx_queue_count() to get the number,
It's also called "used descriptors" in the code.
Also there are Rx buffers provided by SW to allow HW "fill in" received packets, we can call it
"usable Rx buffers" (here "usable" means usable for HW).
May be it is better to stick to Rx descriptor status terminology?
Available - Rx descriptor available to HW to put received packet to
Done - Rx descriptor with received packet reported to Sw
Unavailable - other (e.g. gap which cannot be used or just processed
Done, but not refilled (made available to HW).
Let's define Rx queue "fullness":
Fullness = ready-Rx-buffers/Rxq-size
i.e. number of DONE descriptors divided by RxQ size
On the opposite, we have "emptiness"
Emptiness = usable-Rx-buffers/Rxq-size
i.e. number of AVAIL descriptors divided by RxQ size
Note, that AVAIL != RxQ-size - DONE
HW really knows number of available descriptors by its nature.
It is a space between latest done and latest received on refill.
HW does not know which descriptors are DONE, since some which
are DONE before could be already processed by SW, but not yet
made available again.
Here "fill threshold" describes "fullness", it's not "refill" described in you above words. Because in your
words, "refill" is the opposite, it's filling "usable/free Rx buffers", or "emptiness".
I can only briefly explain how mlx5 works to get LWM, because I'm not a
Firmware guy.
Mlx5 Rx queue is basically RDMA queue. It has two indexes: producer index which
increases when HW fills in packet, consumer index which increases when SW
consumes the packet.
The queue size is known when it's created. The fullness is something like
(producer_index - consumer_index) (I don't consider in wrap-around here).
So mlx5 has the way to get the fullness or emptiness in HW or FW.
Another detail is mlx5 uses the term "LWM"(limit watermark), which describes
"emptiness". When usable-Rx-buffers is below LWM, we trigger an event.
But Thomas think "fullness" is easier to understand, so we use "fullness" in
rte APIs and we'll translate it to LWM in mlx5 PMD.
I may be wrong :)
HW simply does now know fullness and there can't generate any events
based on it. It is a problem on Rx when there is now available
descriptors. I.e. emptiness.
So you think "empty_thresh" would be better?
Or "avail_thresh"?
I'd go for "avail_thresh" since it is consistent with descriptor status
API terminology.
One moment I thought that there is a problem with rx_free_threshold, but
finally realize that there is no problem, since it is in terms of free
descriptors which could be made available (refilled).
Fill threshold is defined as a percentage of Rx queue size with valid
value of [0,99].
Setting fill threshold to 0 means disable it, which is the default.
Add fill threshold configuration and query driver callbacks in eth_dev_ops.
Add command line options to support fill_thresh per-rxq configure.
- Command syntax:
set port <port_id> rxq <rxq_id> fill_thresh <fill_thresh_num>
- Example commands:
To configure fill_thresh as 30% of rxq size on port 1 rxq 0:
testpmd> set port 1 rxq 0 fill_thresh 30
To disable fill_thresh on port 1 rxq 0:
testpmd> set port 1 rxq 0 fill_thresh 0
Signed-off-by: Spike Du <spi...@nvidia.com>
