The setup of the TCP session is handled by the kernel, hence the higher
TTL. Once TCP is established, (e)BGP tends to use a TTL of 1 unless it's a
multihop session.l, or you're using GTSM.
It's expected, and is partly due to the limitations of how sockets are
implemented in Linux.
M
On Fri, 1 Apr 2022, 03:10 Rumen Telbizov, <rumen.telbi...@menlosecurity.com>
wrote:
> Hello Bird users,
>
> First time poster and new subscriber.
> I noticed something strange and wanted to report it here in case this is
> in fact a bug that deserves attention.
>
> I run bird 2.0.7-4.1 on Debian 11.
> I have a BGP section configured as passive that acts as a TCP health-check
> endpoint.
>
> It is as follows:
> *--- cut --*
> protocol bgp HEALTHCHECKv4 {
> hold time 6;
> startup hold time 20;
> connect delay time 3;
> connect retry time 6;
> error wait time 3, 12;
> passive on;
>
> local 100.64.0.5 as 65000;
> neighbor 100.64.0.4 as 65535;
> }
> *--- cut --*
>
>
> What ends up happening on the wire is this:
> *--- cut --*
> 23:15:09.443792 IP (tos 0x0, ttl 254, id 4040, offset 0, flags [DF], proto
> TCP (6), length 60)
> 100.64.0.4.16141 > 100.64.0.5.179: Flags [S], cksum 0xa78f (correct),
> seq 723435095, win 8961, options [mss 8621,sackOK,TS val 3290475421 ecr
> 0,nop,wscale 0], length 0
> 23:15:09.443823 IP (tos 0xc0, ttl 255, id 0, offset 0, flags [DF], proto
> TCP (6), length 60)
> 100.64.0.5.179 > 100.64.0.4.16141: Flags [S.], cksum 0xc8b7 (incorrect
> -> 0x0785), seq 124371865, ack 723435096, win 62643, options [mss
> 8961,sackOK,TS val 2210037294 ecr 3290475421,nop,wscale 7], length 0
> 23:15:09.444437 IP (tos 0x0, ttl 254, id 4041, offset 0, flags [DF], proto
> TCP (6), length 52)
> 100.64.0.4.16141 > 100.64.0.5.179: Flags [.], cksum 0x2550 (correct),
> seq 1, ack 1, win 8961, options [nop,nop,TS val 3290475422 ecr 2210037294],
> length 0
> 23:15:09.444471 IP (tos 0x0, ttl 254, id 4042, offset 0, flags [DF], proto
> TCP (6), length 52)
> 100.64.0.4.16141 > 100.64.0.5.179: Flags [F.], cksum 0x254e (correct),
> seq 1, ack 1, win 8961, options [nop,nop,TS val 3290475423 ecr 2210037294],
> length 0
> 23:15:09.444576 IP (tos 0xc0, *ttl 1*, id 55411, offset 0, flags [DF],
> proto TCP (6), length 99)
> 100.64.0.5.179 > 100.64.0.4.16141: Flags [P.], cksum 0xc8de (incorrect
> -> 0x58b6), seq 1:48, ack 2, win 490, options [nop,nop,TS val 2210037294
> ecr 3290475423], length 47: BGP
> Open Message (1), length: 47
> Version 4, my AS 65000, Holdtime 6s, ID 100.64.0.5
> Optional parameters, length: 18
> Option Capabilities Advertisement (2), length: 16
> Route Refresh (2), length: 0
> Graceful Restart (64), length: 2
> Restart Flags: [none], Restart Time 120s
> 0x0000: 0078
> 32-Bit AS Number (65), length: 4
> 4 Byte AS 65000
> 0x0000: 0000 fde8
> Enhanced Route Refresh (70), length: 0
> no decoder for Capability 70
> Long-lived Graceful Restart (71), length: 0
>
> 23:15:09.444602 IP (tos 0xc0, *ttl 1*, id 55412, offset 0, flags [DF],
> proto TCP (6), length 52)
> 100.64.0.5.179 > 100.64.0.4.16141: Flags [F.], cksum 0xc8af (incorrect
> -> 0x4635), seq 48, ack 2, win 490, options [nop,nop,TS val 2210037294 ecr
> 3290475423], length 0
> 23:15:09.444670 IP (tos 0x0, ttl 64, id 1, offset 0, flags [none], proto
> ICMP (1), length 56)
> 100.64.0.4 > 100.64.0.5: ICMP time exceeded in-transit, length 36
> IP (tos 0xc0, ttl 1, id 55411, offset 0, flags [DF], proto TCP (6),
> length 99)
> 100.64.0.5.179 > 100.64.0.4.16141: [|tcp]
> *--- cut --*
>
> As you can see the TTL on our packets is initially set to 255. At the end
> of the connection
> during the last PUSH and FIN packets all of a sudden bird sets the TTL to
> 1.
>
> I have no ttl security enabled and even if I explicitly disable it the
> problem persists.
> A workaround that I found to work is to pass multihop 20 directive which
> then changes the ttl 1 above to ttl 20
> which alleviates the problem.
>
> Let me know if you need any additional information.
>
> Regards,
> --
> Rumen Telbizov
> Site Reliability Engineer <http://telbizov.com/>
>
