Hi Elias,
Sorry for the delay...
The issue is actually a known issue but ill-documented: because of the way
Mellanox NICs handle jumbo frame, we cannot support chain-buffers efficiently.
You need to configure VPP buffer data size big enough to fit the biggest packet
you expect to receive.
For example, you can add the stanza "buffers { default data-size 8000 }" to
your startup.conf [1]. It is not done by default because it will waste a lot of
memory for all the other interfaces.
Also note that there are some limitations right now regarding "default
data-size" accepted values, eg. setting it to 9000 will crash at initialization.
I'll fix the doc (and add an error counter to make it obvious) and the crash.
Best
ben
[1]
https://fd.io/docs/vpp/master/gettingstarted/users/configuring/startup.html#the-buffers-section
> -----Original Message-----
> From: Elias Rudberg <elias.rudb...@bahnhof.net>
> Sent: mardi 2 juin 2020 12:41
> To: Benoit Ganne (bganne) <bga...@cisco.com>; cho...@chopps.org; vpp-
> d...@lists.fd.io
> Subject: Re: [vpp-dev] ixge and rdma drivers
>
> Hi Ben,
>
> > > (I think there may be a problem with the rdma plugin for larger MTU
> > > values but for MTU < 2000 or so, everything works fine.)
> >
> > It should work, jumbo support was added in the last months. Or do you
> > refer to something else?
>
> I think I mean something else, a problem that I noticed a few weeks ago
> but never had time to report it then. Now I tried again and it can
> still be reproduced with the current master branch.
>
> The setup is that I have one server running VPP doing NAT44 and then I
> have two other servers on inside and outside. This works fine when the
> MTU is 1500. Then I set the MTU to 3000 on all involved interfaces and
> restart VPP. Now it works as longas only small packets are used, but as
> soon as a packet larger than ~2048 bytes appears, VPP stops working.
> (Doing e.g. ping -s 2100 is enough to trigger it.) After that VPP is
> stuck in some kind of error state from which it does not recover, even
> small packets are not forwarded after that.
>
> I tried to investigate further and then it seemed like that what
> happens is that the RDMA_DEVICE_F_ERROR flag is set in
> src/plugins/rdma/input.c which causes the rdma plugin code to get
> stuck, the error flag is never cleared it seems.
>
> The reason why the larger packet size caused an error seems to be that
> the log2_cq_size value used in src/plugins/rdma/input.c is
> log2_cq_size = 11 which corresponds to 2^11 = 2048 bytes which is
> roughly the packet size where the problem appears.
>
> So I got the impression that the rdma plugin is limited to 2^11 = 2048
> bytes MTU due to the log2_cq_size = 11 value. Maybe that can be
> configured somehow? In any case, it seems bad that VPP gets stuck after
> one such error appears, it would be better if it just increased an
> error counter and dropped the packet.
>
> Best regards,
> Elias
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16714): https://lists.fd.io/g/vpp-dev/message/16714
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
