> From: Spike Du [mailto:spi...@nvidia.com] > Sent: Wednesday, 25 May 2022 15.59 > > > From: Morten Brørup <m...@smartsharesystems.com> > > Sent: Wednesday, May 25, 2022 9:40 PM > > > > > From: Spike Du [mailto:spi...@nvidia.com] > > > Sent: Wednesday, 25 May 2022 15.15 > > > > > > > From: Morten Brørup <m...@smartsharesystems.com> > > > > Sent: Wednesday, May 25, 2022 3:00 AM > > > > > > > > > From: Thomas Monjalon [mailto:tho...@monjalon.net] > > > > > Sent: Tuesday, 24 May 2022 17.59 > > > > > > > > > > +Cc people involved in previous versions > > > > > > > > > > 24/05/2022 17:20, Spike Du: > > > > > > LWM(limit watermark) is per RX queue attribute, when RX queue > > > > > fullness reach the LWM limit, HW sends an event to dpdk > > > application. > > > > > > Host shaper can configure shaper rate and lwm-triggered for a > > > host > > > > > port. > > > > > > > > > > > > > > The shaper limits the rate of traffic from host port to wire > > > port. > > > > > > > > From host to wire? It is RX, so you must mean from wire to host. > > > > > > The host shaper is quite private to Nvidia's BlueField 2 NIC. The > NIC > > > is inserted In a server which we call it host-system, and the NIC > has > > > an embedded Arm-system Which does the forwarding. > > > The traffic flows from host-system to wire like this: > > > Host-system generates traffic, send it to Arm-system, Arm sends it > to > > > physical/wire port. > > > So the RX happens between host-system and Arm-system, and the > traffic > > > is host to wire. > > > The shaper also works in a special way: you configure it on > > > Arm-system, but it takes effect On host-sysmem's TX side. > > > > > > > > > > > > > If lwm-triggered is enabled, a 100Mbps shaper is enabled > > > > > automatically when one of the host port's Rx queues receives > LWM > > > event. > > > > > > > > > > > > These two features can combine to control traffic from host > port > > > to > > > > > wire port. > > > > > > > > Again, you mean from wire to host? > > > > > > Pls see above. > > > > > > > > > > > > > The work flow is configure LWM to RX queue and enable lwm- > > > triggered > > > > > flag in host shaper, after receiving LWM event, delay a while > > > > > until > > > RX > > > > > queue is empty , then disable the shaper. We recycle this work > > > > > flow > > > to > > > > > reduce RX queue drops. > > > > > > > > You delay while RX queue gets drained by some other threads, I > > > assume. > > > > > > The PMD thread drains the Rx queue, the PMD receiving as normal, > as > > > the PMD Implementation uses rte interrupt thread to handle LWM > event. > > > > > > > Thank you for the explanation, Spike. It really clarifies a lot! > > > > If this patch is intended for DPDK running on the host-system, then > the LWM > > attribute is associated with a TX queue, not an RX queue. The packets > are > > egressing from the host-system, so TX from the host-system's > perspective. > > > > Otherwise, if this patch is for DPDK running on the embedded ARM- > system, > > it should be highlighted somewhere. > > The host-shaper patch is running on ARM-system, I think in that patch I > have some explanation in mlx5.rst. > The LWM patch is common and should work on any Rx queue(right now mlx5 > doesn't support Hairpin Rx queue and shared Rx queue). > On ARM-system, we can use it to monitor traffic from host(representor > port) or from wire(physical port). > LWM can also work on host-system if there is DPDK running, for example > it can monitor traffic from Arm-system to host-system.
OK. Then I get it! I was reading the patch description wearing my host-system glasses, and thus got very confused. :-) > > > > > > > > > > > Surely, the excess packets must be dropped somewhere, e.g. by the > > > shaper? > > > > I guess the shaper doesn't have to drop any packets, but the host- > system will > > simply be unable to put more packets into the queue if it runs full. > > > > When LWM event happens, the host-shaper throttles traffic from host- > system to Arm-system. Yes, the shaper doesn't drop pkts. > Normally the shaper is small and if PMD thread on Arm keeps working, Rx > queue is dropless. > But if PMD thread doesn't receive fast enough, or even with a small > shaper but host-system is sending some burst, Rx queue may still drop > on Arm. > Anyway even sometimes drop still happens, the cooperation of host- > shaper and LWM greatly reduce the Rx drop on Arm. Thanks for elaborating. And yes, shapers are excellent for many scenarios.
