Hoi,
I know this is a long shot, but this afternoon I upgraded one of my
routers at AS50869 from VPP 24.10 to VPP 25.10.0 (with the LinuxCP fix).
Shortly there-after, two internet exchanges (both with a /23 peeringlan)
complained that the router was flooding ARP requests.
I could not see these in Linux CP, but they were visible in the Internet
Exchange when looking at a packet dump.
I could however see the ARP replies from the folks my router was
flooding, like so:
root@nlams0:/etc/bird/ebgp/groups# time tcpdump -evni speedix arp
16:08:46.638960 44:4c:a8:c7:4a:33 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.223.39
is-at 44:4c:a8:c7:4a:33, length 46
16:08:46.881844 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:46.886507 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:46.902967 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:46.905873 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:46.940812 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:46.973942 44:4c:a8:c7:4a:33 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.223.39
is-at 44:4c:a8:c7:4a:33, length 46
16:08:46.983844 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.003539 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.010575 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.030974 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.069999 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.123048 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.134061 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.158991 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
16:08:47.159000 c4:ca:2b:69:c8:f7 > b8:59:9f:e2:0a:9f, ethertype ARP
(0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 185.1.222.21
is-at c4:ca:2b:69:c8:f7, length 46
In this network, the VPP router is b8:59:9f:e2:0a:9f and the router that
was being flooded is at c4:ca:2b:69:c8:f7. They saw the traffic also. It
immediately stopped when I set a static neighbor entry in Linux. I took
a trace from dpdk-input but it did not reveal any outbound ARP traffic
(which makes sense). It did however show the ARP replies.
Could it be that between 24.10 and 25.10 release, something changed in
the ARP handling that might trigger an ARP flood from within arp
request/reply/ip-neighbor code? I'm hoping somebody can remember any
changes, I scanned over a bunch of changes but a year is a long time and
bisecting on an internet exchange is impractical. I may be able to repro
this behavior in a lab, but before I go deeper: does this ARP flooding
ring a bell for anybody ?
groet,
Pim
--
Pim van Pelt <[email protected]>
PBVP1-RIPE https://ipng.ch/
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#26582): https://lists.fd.io/g/vpp-dev/message/26582
Mute This Topic: https://lists.fd.io/mt/116519525/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
