[vpp-dev] More information about our old VPP VRRP problem, seems to be a MAC issue in VPP

[Peter Olsson via lists.fd.io]

Hello!

We may have found the cause of this problem, but don't know how to fix it.

When VRRP primary is switched between routers, the VPP router
hangs on to the old MAC address for the virtual IP inside VPP.
Ie the VPP router is primary VRRP in keepalived, but still keeps
the MAC address for the other router in the VPP neighbor table.

And then the VPP router stops answering ARP requests for the
virtual IP, and ICMP ping toward the virtual IP gets very high
response time every ten seconds, seemingly corresponding to
Gracious ARP interval.

When we manually delete the MAC entry for the virtual IP in VPP,
with set ip neighbor delete, ARP traffic starts working.

The VPP router is now Ubuntu 24.04.3 with VPP 25.06-release,
and running keepalived with unicast_src_ip and unicast_peer.

The other keepalived router is an old Debian, not running VPP.


Our startup.conf:

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen /run/vpp/cli.sock
  gid vpp
  startup-config /home/pol/bin/start_vpp
}
api-trace {
  on
}
api-segment {
  gid vpp
}
socksvr {
  default
}
memory {
        main-heap-size 2G
        main-heap-page-size default-hugepage
}
cpu {
        workers 4
}
dpdk {
        dev 0000:01:00.0
        dev 0000:01:00.1
        dev 0000:02:00.0
        dev 0000:02:00.1
}
plugins {
        plugin default { disable }
        plugin dpdk_plugin.so { enable }
        plugin acl_plugin.so { enable }
        plugin lacp_plugin.so { enable }
        plugin flowprobe_plugin.so { enable }
        plugin vxlan_plugin.so { enable }
        plugin linux_cp_plugin.so { enable }
        plugin linux_nl_plugin.so { enable }
        plugin ping_plugin.so { enable }
}
statseg {
    size 1G
    per-node-counters on
}
l2fib {
    table-size 2048M
    num-buckets 1048576
}


And this is the start_vpp file:

lcp lcp-sync on
lcp lcp-auto-subint on

create loopback interface instance 0
lcp create loop0 host-if loop0
set interface ip address loop0 y.y.y.113/32
set interface ip address loop0 y:y:y::113/128
set interface mtu packet 1500 loop0
set interface state loop0 up

create bond mode lacp load-balance l23

set interface mtu packet 9000 TwentyFiveGigabitEthernet1/0/0
set interface mtu packet 9000 TwentyFiveGigabitEthernet1/0/1
set interface mtu packet 9000 TwentyFiveGigabitEthernet2/0/0
set interface mtu packet 9000 TwentyFiveGigabitEthernet2/0/1
set interface mtu packet 9000 BondEthernet0

bond add BondEthernet0 TwentyFiveGigabitEthernet1/0/0
bond add BondEthernet0 TwentyFiveGigabitEthernet1/0/1
bond add BondEthernet0 TwentyFiveGigabitEthernet2/0/0
bond add BondEthernet0 TwentyFiveGigabitEthernet2/0/1

set interface state TwentyFiveGigabitEthernet1/0/0 up
set interface state TwentyFiveGigabitEthernet1/0/1 up
set interface state TwentyFiveGigabitEthernet2/0/0 up
set interface state TwentyFiveGigabitEthernet2/0/1 up
set interface state BondEthernet0 up

lcp create BondEthernet0 host-if vpp0

create sub-interfaces BondEthernet0 308
set interface ip address BondEthernet0.308 x.x.x.29/28
set interface mtu packet 1500 BondEthernet0.308
set interface state BondEthernet0.308 up

And more VLANs.


And keepalived.conf:

global_defs {
  vrrp_garp_master_refresh 5
  vrrp_garp_master_refresh_repeat 5
  vrrp_garp_lower_prio_repeat 5
  vrrp_garp_interval 0.01
  vrrp_gna_interval 0.01
  vrrp_garp_extra_if all 100
}

vrrp_instance VI_3 {
 state MASTER
 interface vpp0.308
 virtual_router_id 1
 priority 150
 #use_vmac (We have been labbing with use_vmac, doesn't help)
 authentication {
  auth_type PASS
  auth_pass xxx
 }
 unicast_src_ip x.x.x.29
 unicast_peer {
  x.x.x.30
 }
 virtual_ipaddress {
  x.x.x.17/28  (Here we have been labbing with /32 or no netmask, doesn't help)
 }
}

And more VLANs.

(The garp values are probably not ideal, I have been trying
lots of different garp parameters and values without success.)



Are we missing some configuration or is there a problem in VPP?

Can I provide anything else to help troubleshooting?

Thanks!

Peter Olsson

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#26287): https://lists.fd.io/g/vpp-dev/message/26287
Mute This Topic: https://lists.fd.io/mt/114803330/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-