On Wed, 6 Mar 2024 16:14:51 +0000
Ferruh Yigit <ferruh.yi...@amd.com> wrote:
> On 2/29/2024 5:56 PM, Stephen Hemminger wrote:
> > The tap device needs to exchange file descriptors for tx and rx.
> > But the EAL MP layer has limit of 8 file descriptors per message.
> > The ideal resolution would be to increase the number of file
> > descriptors allowed for rte_mp_sendmsg(), but this would break
> > the ABI. Workaround the constraint by breaking into multiple messages.
> >
> > Do not hide errors about MP message failures.
> >
> > Signed-off-by: Stephen Hemminger <step...@networkplumber.org>
> > ---
> > drivers/net/tap/rte_eth_tap.c | 40 +++++++++++++++++++++++++++++------
> > 1 file changed, 33 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> > index 69d9da695bed..df18c328f498 100644
> > --- a/drivers/net/tap/rte_eth_tap.c
> > +++ b/drivers/net/tap/rte_eth_tap.c
> > @@ -863,21 +863,44 @@ tap_mp_req_on_rxtx(struct rte_eth_dev *dev)
> > msg.fds[fd_iterator++] = process_private->txq_fds[i];
> > msg.num_fds++;
> > request_param->txq_count++;
> > +
> > + /* Need to break request into chunks */
> > + if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
> > + err = rte_mp_sendmsg(&msg);
> > + if (err < 0)
> > + goto fail;
> > +
> > + fd_iterator = 0;
> > + msg.num_fds = 0;
> > + request_param->txq_count = 0;
> > + }
> > }
> > for (i = 0; i < dev->data->nb_rx_queues; i++) {
> > msg.fds[fd_iterator++] = process_private->rxq_fds[i];
> > msg.num_fds++;
> > request_param->rxq_count++;
> > +
> > + if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
> > + err = rte_mp_sendmsg(&msg);
> > + if (err < 0)
> > + goto fail;
> > +
> > + fd_iterator = 0;
> > + msg.num_fds = 0;
> > + request_param->rxq_count = 0;
> > + }
> > }
>
> Hi Stephen,
>
> Did you able to verify with more than 4 queues?
>
> As far as I can see, in the secondary counterpart of the
> 'rte_mp_sendmsg()', each time secondary index starts from 0, and
> subsequent calls overwrites the fds in secondary.
> So practically still only 4 queues works.
I got 4 queues setup, but looks like they are trash in secondary.
Probably best to revert this and fix it by bumping RTE_MP_MAX_FD_NUM.
This is better, but does take some ABI issue handling.
