Hello Andre,
I'd suggest upgrading to the latest stable release.
For wrong ECMP: are you sure you have different metrics in Linux? can you
show the output of ip route?
For stale entries: did you adjust sysctl to support big buffers for netlink
socket? Do you see in lcp log messages about re-synchronization?
On Tue, 5 Aug 2025 at 14:41, Andre Nathan via lists.fd.io <andre=
digirati.com...@lists.fd.io> wrote:
> Hello
>
> I have a two-router setup where one is running VPP with LCP and the
> other is a regular Linux box. I took down the Linux router to reinstall
> it with VPP and noticed the following issue on the VPP router.
>
> For context, the routers each have two BGP sessions with uplink
> providers and an iBGP session with each other, for redundancy. When I
> turned off the Linux router, I noticed that the routes learned via the
> iBGP session remained in the VPP FIB.
>
> In the output below, a.b.c.d is the IP of the Linux router and x.y.z.w
> is the IP of the uplink provider's router.
>
> # vppctl show ip fib 138.117.103.55/32
> ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel
> ] epoch:0 flags:none locks:[adjacency:1, default-route:1, lcp-rt:1, ]
> 138.117.102.0/23 fib:0 index:252584 locks:2
> lcp-rt-dynamic refs:1 src-flags:added,contributing,active,
> path-list:[352] locks:4258 flags:shared,popular, uPRF-list:70 len:2
> itfs:[3, 7, ]
> path:[65] pl-index:352 ip4 weight=1 pref=32 attached-nexthop:
> oper-flags:resolved,
> a.b.c.d BondEthernet0
> [@0]: ipv4 via a.b.c.d BondEthernet0: mtu:1500 next:8 flags:[]
> 5a80bd2968e03cfdfe9ec4e40800
> path:[374] pl-index:352 ip4 weight=1 pref=32 attached-nexthop:
> oper-flags:resolved,
> x.y.z.w uplink1
> [@0]: ipv4 via x.y.z.w uplink1: mtu:1500 next:5 flags:[]
> b49691b2a989b49691b2a6200800
>
> However in bird this is the available route:
>
> bird> show route for 138.117.103.55
> Table master4:
> 138.117.102.0/23 unicast [uplink2_ipv4 2025-07-24] * (100) [AS262387i]
> via k.l.m.n on uplink2
>
> So in fact both routes in the VPP FIB were stale, and the correct route
> wasn't installed (The uplink2 interface is not compatible with DPDK so
> it's managed by Linux, but it's moved to the VPP namespace so that
> forwarding works).
>
> I tried removing the route manually with "vppctl ip route del
> 138.117.102.0/23 via a.b.c.d" but that had no effect.
>
> I've found this thread about LCP routes not being deleted [1] but not
> much else.
>
> Has anyone ever seen this behavior, and is there a workaround? The
> output above is just an example for a single prefix, but it seems all
> routes previously learned via the iBGP session were still in the FIB,
> and likely many routes from the other BGP sessions were stale too.
>
> My configuration files follow. I'm using VPP 24.04 on this router.
>
> # cat /etc/vpp/startup.conf
> unix {
> gid vpp
> nodaemon
> cli-listen /run/vpp/cli.sock
> exec /etc/vpp/bootstrap.vpp
> }
>
> api-trace { on }
> api-segment { gid vpp }
> socksvr { default }
>
> memory {
> main-heap-size 2G
> main-heap-page-size default-hugepage
> }
>
> cpu {
> main-core 1
> corelist-workers 3,5,7,9,11,13,15,17,19
> }
>
> buffers {
> buffers-per-numa 128000
> default data-size 16384
> page-size default-hugepage
> }
>
> statseg {
> size 1G
> page-size default-hugepage
> per-node-counters off
> }
>
> plugins {
> plugin default { disable }
> plugin dpdk_plugin.so { enable }
> plugin linux_nl_plugin.so { enable }
> plugin linux_cp_plugin.so { enable }
> }
>
> logging {
> default-log-level warn
> default-syslog-log-level warn
> }
>
> dpdk {
> uio-driver vfio-pci
>
> dev 0000:41:00.0 {
> name uplink1
> }
> dev 0000:05:00.0 {
> name ext0
> }
> dev 0000:05:00.1 {
> name ext1
> }
> }
>
> linux-cp {
> default netns vpp
> lcp-sync
> lcp-auto-subint
> }
>
> # cat /etc/vpp/bootstrap.vpp
> lcp create uplink1 host-if uplink1
> set interface state uplink1 up
> set interface ip address uplink1 x.y.z.z/30
> set interface ip address uplink1 x:y:z::z/126
>
> lcp create ext0 host-if ext0
> lcp create ext1 host-if ext1
>
> create bond mode active-backup
> bond add BondEthernet0 ext0
> bond add BondEthernet0 ext1
>
> set interface state ext0 up
> set interface state ext1 up
>
> lcp create BondEthernet0 host-if external
> set interface state BondEthernet0 up
> set interface ip address BondEthernet0 a.b.c.e/22
> set interface ip address BondEthernet0 a:b:c::e/48
>
> Thanks in advance,
> Andre
>
> [1] https://lists.fd.io/g/vpp-dev/topic/111508185
>
>
>
>
--
Best regards
Stanislav Zaikin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#26248): https://lists.fd.io/g/vpp-dev/message/26248
Mute This Topic: https://lists.fd.io/mt/114547460/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
