On 5/25/22 17:16, Morten Brørup wrote:
From: Spike Du [mailto:spi...@nvidia.com]
Sent: Wednesday, 25 May 2022 15.59
From: Morten Brørup <m...@smartsharesystems.com>
Sent: Wednesday, May 25, 2022 9:40 PM
From: Spike Du [mailto:spi...@nvidia.com]
Sent: Wednesday, 25 May 2022 15.15
From: Morten Brørup <m...@smartsharesystems.com>
Sent: Wednesday, May 25, 2022 3:00 AM
From: Thomas Monjalon [mailto:tho...@monjalon.net]
Sent: Tuesday, 24 May 2022 17.59
+Cc people involved in previous versions
24/05/2022 17:20, Spike Du:
LWM(limit watermark) is per RX queue attribute, when RX queue
fullness reach the LWM limit, HW sends an event to dpdk
application.
Host shaper can configure shaper rate and lwm-triggered for a
host
port.
The shaper limits the rate of traffic from host port to wire
port.
From host to wire? It is RX, so you must mean from wire to host.
The host shaper is quite private to Nvidia's BlueField 2 NIC. The
NIC
is inserted In a server which we call it host-system, and the NIC
has
an embedded Arm-system Which does the forwarding.
The traffic flows from host-system to wire like this:
Host-system generates traffic, send it to Arm-system, Arm sends it
to
physical/wire port.
So the RX happens between host-system and Arm-system, and the
traffic
is host to wire.
The shaper also works in a special way: you configure it on
Arm-system, but it takes effect On host-sysmem's TX side.
If lwm-triggered is enabled, a 100Mbps shaper is enabled
automatically when one of the host port's Rx queues receives
LWM
event.
These two features can combine to control traffic from host
port
to
wire port.
Again, you mean from wire to host?
Pls see above.
The work flow is configure LWM to RX queue and enable lwm-
triggered
flag in host shaper, after receiving LWM event, delay a while
until
RX
queue is empty , then disable the shaper. We recycle this work
flow
to
reduce RX queue drops.
You delay while RX queue gets drained by some other threads, I
assume.
The PMD thread drains the Rx queue, the PMD receiving as normal,
as
the PMD Implementation uses rte interrupt thread to handle LWM
event.
Thank you for the explanation, Spike. It really clarifies a lot!
If this patch is intended for DPDK running on the host-system, then
the LWM
attribute is associated with a TX queue, not an RX queue. The packets
are
egressing from the host-system, so TX from the host-system's
perspective.
Otherwise, if this patch is for DPDK running on the embedded ARM-
system,
it should be highlighted somewhere.
The host-shaper patch is running on ARM-system, I think in that patch I
have some explanation in mlx5.rst.
The LWM patch is common and should work on any Rx queue(right now mlx5
doesn't support Hairpin Rx queue and shared Rx queue).
On ARM-system, we can use it to monitor traffic from host(representor
port) or from wire(physical port).
LWM can also work on host-system if there is DPDK running, for example
it can monitor traffic from Arm-system to host-system.
OK. Then I get it! I was reading the patch description wearing my host-system
glasses, and thus got very confused. :-)
The description in cover letter is very misleading for me as
well. It is not a problem right now after long detailed
explanations. Hopefully there is no such problem in suggested
ethdev documentation. I'll reread it carefully before applying
when time comes.
Surely, the excess packets must be dropped somewhere, e.g. by the
shaper?
I guess the shaper doesn't have to drop any packets, but the host-
system will
simply be unable to put more packets into the queue if it runs full.
When LWM event happens, the host-shaper throttles traffic from host-
system to Arm-system. Yes, the shaper doesn't drop pkts.
Normally the shaper is small and if PMD thread on Arm keeps working, Rx
queue is dropless.
But if PMD thread doesn't receive fast enough, or even with a small
shaper but host-system is sending some burst, Rx queue may still drop
on Arm.
Anyway even sometimes drop still happens, the cooperation of host-
shaper and LWM greatly reduce the Rx drop on Arm.
Thanks for elaborating. And yes, shapers are excellent for many scenarios.
