If you think HW knows "emptiness", and we want to stick to DPDK descriptor terms. Can we use the name "avail_thresh?" When HW sees available descriptors is under this threshold, an event is triggered.
> -----Original Message----- > From: Andrew Rybchenko <andrew.rybche...@oktetlabs.ru> > Sent: Tuesday, June 7, 2022 1:16 AM > To: Spike Du <spi...@nvidia.com>; Matan Azrad <ma...@nvidia.com>; > Slava Ovsiienko <viachesl...@nvidia.com>; Ori Kam <or...@nvidia.com>; > NBU-Contact-Thomas Monjalon (EXTERNAL) <tho...@monjalon.net>; > Wenzhuo Lu <wenzhuo...@intel.com>; Beilei Xing <beilei.x...@intel.com>; > Bernard Iremonger <bernard.iremon...@intel.com>; Ray Kinsella > <m...@ashroe.eu>; Neil Horman <nhor...@tuxdriver.com> > Cc: step...@networkplumber.org; m...@smartsharesystems.com; > dev@dpdk.org; Raslan Darawsheh <rasl...@nvidia.com> > Subject: Re: [PATCH v4 3/7] ethdev: introduce Rx queue based fill threshold > > External email: Use caution opening links or attachments > > > On 6/6/22 16:16, Spike Du wrote: > > Hi Andrew, > > Please see below for "fill threshold" concept, I'm ok with other > comments about code. > > > > Regards, > > Spike. > > > > > >> -----Original Message----- > >> From: Andrew Rybchenko <andrew.rybche...@oktetlabs.ru> > >> Sent: Saturday, June 4, 2022 8:46 PM > >> To: Spike Du <spi...@nvidia.com>; Matan Azrad <ma...@nvidia.com>; > >> Slava Ovsiienko <viachesl...@nvidia.com>; Ori Kam <or...@nvidia.com>; > >> NBU-Contact-Thomas Monjalon (EXTERNAL) <tho...@monjalon.net>; > Wenzhuo > >> Lu <wenzhuo...@intel.com>; Beilei Xing <beilei.x...@intel.com>; > >> Bernard Iremonger <bernard.iremon...@intel.com>; Ray Kinsella > >> <m...@ashroe.eu>; Neil Horman <nhor...@tuxdriver.com> > >> Cc: step...@networkplumber.org; m...@smartsharesystems.com; > >> dev@dpdk.org; Raslan Darawsheh <rasl...@nvidia.com> > >> Subject: Re: [PATCH v4 3/7] ethdev: introduce Rx queue based fill > >> threshold > >> > >> External email: Use caution opening links or attachments > >> > >> > >> On 6/3/22 15:48, Spike Du wrote: > >>> Fill threshold describes the fullness of a Rx queue. If the Rx queue > >>> fullness is above the threshold, the device will trigger the event > >>> RTE_ETH_EVENT_RX_FILL_THRESH. > >> > >> Sorry, I'm not sure that I understand. As far as I know the process > >> to add more Rx buffers to Rx queue is called 'refill' in many > >> drivers. So fill level is a number (or percentage) of free buffers in an Rx > queue. > >> If so, fill threashold should be a minimum fill level and below the > >> level we should generate an event. > >> > >> However reading the first paragraph of the descrition it looks like > >> you mean oposite thing - a number (or percentage) of ready Rx buffers > >> with received packets. > >> > >> I think that the term "fill threshold" is suggested by me, but I did > >> it with mine understanding of the added feature. Now I'm confused. > >> > >> Moreover, I don't understand how "fill threshold" could be in terms > >> of ready Rx buffers. HW simply don't really know when ready Rx > >> buffers are processed by SW. So, HW can't say for sure how many ready > >> Rx buffers are pending. It could be calculated as Rx queue size minus > >> number of free Rx buffers, but it is imprecise. First of all not all Rx > descriptors could be used. > >> Second, HW ring size could differ queue size specified in SW. > >> Queue size specified in SW could just limit maximum nubmer of free Rx > >> buffers provided by the driver. > >> > > > > Let me use other terms because "fill"/"refill" is also ambiguous to me. > > In a RX ring, there are Rx buffers with received packets, you call it > > "ready Rx buffers", there is a RTE api rte_eth_rx_queue_count() to get the > number, It's also called "used descriptors" in the code. > > Also there are Rx buffers provided by SW to allow HW "fill in" received > packets, we can call it "usable Rx buffers" (here "usable" means usable for > HW). > > May be it is better to stick to Rx descriptor status terminology? > Available - Rx descriptor available to HW to put received packet to Done - Rx > descriptor with received packet reported to Sw Unavailable - other (e.g. gap > which cannot be used or just processed Done, but not refilled (made > available to HW). > > > Let's define Rx queue "fullness": > > Fullness = ready-Rx-buffers/Rxq-size > > i.e. number of DONE descriptors divided by RxQ size > > > On the opposite, we have "emptiness" > > Emptiness = usable-Rx-buffers/Rxq-size > > i.e. number of AVAIL descriptors divided by RxQ size Note, that AVAIL != > RxQ-size - DONE > > HW really knows number of available descriptors by its nature. > It is a space between latest done and latest received on refill. > > HW does not know which descriptors are DONE, since some which are DONE > before could be already processed by SW, but not yet made available again. > > > > Here "fill threshold" describes "fullness", it's not "refill" described in > > you > above words. Because in your words, "refill" is the opposite, it's filling > "usable/free Rx buffers", or "emptiness". > > > > I can only briefly explain how mlx5 works to get LWM, because I'm not a > Firmware guy. > > Mlx5 Rx queue is basically RDMA queue. It has two indexes: producer index > which increases when HW fills in packet, consumer index which increases > when SW consumes the packet. > > The queue size is known when it's created. The fullness is something like > (producer_index - consumer_index) (I don't consider in wrap-around here). > > So mlx5 has the way to get the fullness or emptiness in HW or FW. > > Another detail is mlx5 uses the term "LWM"(limit watermark), which > describes "emptiness". When usable-Rx-buffers is below LWM, we trigger an > event. > > But Thomas think "fullness" is easier to understand, so we use "fullness" in > rte APIs and we'll translate it to LWM in mlx5 PMD. > > HW simply does now know fullness and there can't generate any events > based on it. It is a problem on Rx when there is now available descriptors. > I.e. > emptiness. > > >>> Fill threshold is defined as a percentage of Rx queue size with > >>> valid value of [0,99]. > >>> Setting fill threshold to 0 means disable it, which is the default. > >>> Add fill threshold configuration and query driver callbacks in > eth_dev_ops. > >>> Add command line options to support fill_thresh per-rxq configure. > >>> - Command syntax: > >>> set port <port_id> rxq <rxq_id> fill_thresh <fill_thresh_num> > >>> > >>> - Example commands: > >>> To configure fill_thresh as 30% of rxq size on port 1 rxq 0: > >>> testpmd> set port 1 rxq 0 fill_thresh 30 > >>> > >>> To disable fill_thresh on port 1 rxq 0: > >>> testpmd> set port 1 rxq 0 fill_thresh 0 > >>> > >>> Signed-off-by: Spike Du <spi...@nvidia.com> > > [snip]
