On 10/18/21 3:59 PM, Xueming Li wrote: > In current DPDK framework, each Rx queue is pre-loaded with mbufs to > save incoming packets. For some PMDs, when number of representors scale > out in a switch domain, the memory consumption became significant. > Polling all ports also leads to high cache miss, high latency and low > throughput. > > This patch introduce shared Rx queue. Ports in same Rx domain and > switch domain could share Rx queue set by specifying non-zero sharing > group in Rx queue configuration. > > Shared Rx queue is identified by share_rxq field of Rx queue > configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using > same shared Rx queue ID. > > No special API is defined to receive packets from shared Rx queue. > Polling any member port of a shared Rx queue receives packets of that > queue for all member ports, port_id is identified by mbuf->port. PMD is > responsible to resolve shared Rx queue from device and queue data. > > Shared Rx queue must be polled in same thread or core, polling a queue > ID of any member port is essentially same. > > Multiple share groups are supported. Device should support mixed > configuration by allowing multiple share groups and non-shared Rx queue > on one port. > > Example grouping and polling model to reflect service priority: > Group1, 2 shared Rx queues per port: PF, rep0, rep1 > Group2, 1 shared Rx queue per port: rep2, rep3, ... rep127 > Core0: poll PF queue0 > Core1: poll PF queue1 > Core2: poll rep2 queue0 > > PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE. > > PMD is responsible for shared Rx queue consistency checks to avoid > member port's configuration contradict to each other. > > Signed-off-by: Xueming Li <xuemi...@nvidia.com>
with few nits below: Reviewed-by: Andrew Rybchenko <andrew.rybche...@oktetlabs.ru> [snip] > diff --git a/doc/guides/prog_guide/switch_representation.rst > b/doc/guides/prog_guide/switch_representation.rst > index ff6aa91c806..fe89a7f5c33 100644 > --- a/doc/guides/prog_guide/switch_representation.rst > +++ b/doc/guides/prog_guide/switch_representation.rst > @@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for > applications. > .. [1] `Ethernet switch device driver model (switchdev) > <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_ > > +- For some PMDs, memory usage of representors is huge when number of > + representor grows, mbufs are allocated for each descriptor of Rx queue. > + Polling large number of ports brings more CPU load, cache miss and > + latency. Shared Rx queue can be used to share Rx queue between PF and > + representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in > + device info is used to indicate the capability. Setting non-zero share > + group in Rx queue configuration to enable share, share_qid is used to > + identifiy the shared Rx queue in group. Polling any member port can identifiy -> identify > + receive packets of all member ports in the group, port ID is saved in > + ``mbuf.port``. > + > Basic SR-IOV > ------------ > > diff --git a/doc/guides/rel_notes/release_21_11.rst > b/doc/guides/rel_notes/release_21_11.rst > index d5435a64aa1..2143e38ff11 100644 > --- a/doc/guides/rel_notes/release_21_11.rst > +++ b/doc/guides/rel_notes/release_21_11.rst > @@ -75,6 +75,12 @@ New Features > operations. > * Added multi-process support. > > +* **Added ethdev shared Rx queue support.** > + > + * Added new device capability flag and rx domain field to switch info. rx -> Rx > + * Added share group and share queue ID to Rx queue configuration. > + * Added testpmd support and dedicate forwarding engine. > + > * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** > > Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and [snip]
