Hi, I have the most strange issue on a setup of 2 pairs of connected
back to back XL710 cards.
It seems that frames above 1510 bytes are being filtered out by an i40e
PMD receiver.
The same setup works perfectly when I use Linux drivers on both sides
but when I use a PMD on one side and a Linux driver on the other - the
issue above occurs.
i40e PMD statistics show nothing: all counters stay zero.
The MFS field in PRTGL_SAH register has the expected (reset) value: 0x2600.
I even tried to disable all Tx offload capabilities on the Linux side -
still no success.
Could u, guys clarify to be what is going on and what I may be doing wrong?
Pls., let me know if u need any additional info about the setup.
thanks in advance,
vlad
How to reproduce the issue:
I used the in tree apps from examples/multi_process/client_server_mp:
On the DPDK box:
1. Bound both NICs to DPDK's UIO.
2. Configured 1024 huge pages on each NUMA Node (there are two NUMA
Nodes in total).
$ cat /proc/meminfo | grep Huge
AnonHugePages: 0 kB
HugePages_Total: 2048
HugePages_Free: 292
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
3. Compiled mp_server and mp_client example applications.
4. Run: sudo ./build/mp_server -c 6 -n 4 -- -p 0x3 -n 1
5. Run: sudo ./build/mp_client -c 8 -n 4 --proc-type=auto -- -n 0
On the "other side" box (with the Linux device drivers):
1. configure a static arp:
$ arp 192.169.10.118
Address HWtype HWaddress Flags
Mask Iface
192.169.10.118 ether 68:05:ca:2d:39:88 CM
ens3
2. Set the MTU on ens3 to 3000 (see below).
3. Use *ping* application for sending frames of different sizes:
1. "ping 192.169.10.118 -s 1400" makes both Rx and Tx counters on
the DPDK side increment.
2. "ping 192.169.10.118 -s 1500" - Rx counter on the DPDK side
doesn't increment.
The DPDK PMD driven setup description:
* Two XL710 single port cards connected to XL710 cards in a different
server back to back.
$ lspci | grep Ether
---snip---
05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 01)
83:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 01)
---snip---
$ ethtool -i ens288
driver: i40e
version: 1.3.2-k
firmware-version: f4.22.26225 a1.1 n4.24 e13fd
bus-info: 0000:83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
$ ethtool -i ens785
driver: i40e
version: 1.3.2-k
firmware-version: f4.22.26225 a1.1 n4.24 e13fd
bus-info: 0000:05:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
$ ifconfig
---snip---
ens288: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 68:05:ca:2d:3a:00 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens785: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 68:05:ca:2d:39:88 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
---snip---
$ cat /etc/redhat-release
Fedora release 21 (Twenty One)
* DPDK git HEAD 7173acefc7cfdfbbb9b91fcba1c9a67adb4c07c9
The Linux drivers driven setup description:
* Two XL710 single port cards connected to XL710 cards in a different
server back to back.
$ lspci | grep Ether
---snip---
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 01)
83:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 01)
---snip---
$ ethtool -i ens3
driver: i40e
version: 1.2.6-k
firmware-version: f4.1 a1.1 n04.10 e80001121
bus-info: 0000:83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
$ ethtool -i ens5
driver: i40e
version: 1.2.6-k
firmware-version: f4.1 a1.1 n04.10 e80001121
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
$ ifconfig
---snip---
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 3000
inet 192.169.10.199 netmask 255.255.255.0 broadcast
192.169.10.255
ether 68:05:ca:27:d3:72 txqueuelen 1000 (Ethernet)
RX packets 69 bytes 30155 (29.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 746 bytes 820489 (801.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 1.1.1.2 netmask 255.255.255.0 broadcast 1.1.1.255
inet6 fe80::6a05:caff:fe27:d3ea prefixlen 64 scopeid 0x20<link>
ether 68:05:ca:27:d3:ea txqueuelen 1000 (Ethernet)
RX packets 523572634 bytes 34869102046 (32.4 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 523254659 bytes 153342981238 (142.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
---snip---
$ cat /etc/redhat-release
Fedora release 21 (Twenty One)_
_$ modinfo i40e
filename:
/lib/modules/4.0.6-200.fc21.x86_64/kernel/drivers/net/ethernet/intel/i40e/i40e.ko.xz
version: 1.2.6-k
license: GPL
description: Intel(R) Ethernet Connection XL710 Network Driver
author: Intel Corporation, <e1000-devel at lists.sourceforge.net>
srcversion: 14800421FE3354248473729
alias: pci:v00008086d00001586sv*sd*bc*sc*i*
alias: pci:v00008086d00001585sv*sd*bc*sc*i*
alias: pci:v00008086d00001584sv*sd*bc*sc*i*
alias: pci:v00008086d00001583sv*sd*bc*sc*i*
alias: pci:v00008086d00001581sv*sd*bc*sc*i*
alias: pci:v00008086d00001580sv*sd*bc*sc*i*
alias: pci:v00008086d0000157Fsv*sd*bc*sc*i*
alias: pci:v00008086d00001574sv*sd*bc*sc*i*
alias: pci:v00008086d00001572sv*sd*bc*sc*i*
depends: ptp,vxlan
intree: Y
vermagic: 4.0.6-200.fc21.x86_64 SMP mod_unload
signer: Fedora kernel signing key
sig_key: F9:FE:41:30:BC:4F:80:F6:C4:1E:A1:E1:5C:73:37:9B:B8:BA:C8:4C
sig_hashalgo: sha256
parm: debug:Debug level (0=none,...,16=all) (int)
_
_
