Re: IPSec tcp session stalling

Michael VInce Sat, 22 Oct 2005 22:37:29 -0700

I am using FAST_IPSEC on a multi subnet VPN with the guys on other sidehaving Check Point VPN / Firewall.Its a VPN that does almost non stop usage, the people on the other sidehave 24 monitoring utils on it and its never had a problem.Its on 5.3 i386, and I fear to upgrade it, when it comes to VPN Ibelieve in the rule if its not broke don't fix it.

When I think about if I haven't had much luck when trying regular IPSECdespite docs saying its better supported? But then again I never gave ita good shot, FAST_IPSEC just sounded 'faster'


Mike

Volker wrote:

Max & Co:

I've just seen I'm using kernel config 'options IPSEC' on both machines.
Should I try 'options FAST_IPSEC'? Would take some hours for kernel
recompile. Does the code IPSEC / FAST_IPSEC make a difference (even
while having not hardware crypto accelerator)?

May I use FAST_IPSEC even without any hw-crypto devices? While reading
`man fast_ipsec' I would think it depends on a hw-crypto device...

Please tell me if we should check IPSEC / FAST_IPSEC and I'll start a
recompile.

Volker


On 2005-10-23 00:40, Max Laier wrote:

To try something else: Could you guys try to disable SACK on the machinesinvolved? I haven't looked at the dumps as of yet, but that's one simpletest that might help to identify the problem.


sysctl net.inet.tcp.sack.enable=0

On Sunday 23 October 2005 02:23, Volker wrote:

Michael,

I not that sure if I'm right in checking what you suggested but when
trying to do ping hostB from hostA with oversized packets through the
IPSec tunnel by:

# ping -c 10 -s 12000 10.128.6.1

I'm getting replies easily.

While doing that and tcpdump'ing the gif interface, I'm seeing the
fragmented packets coming in properly.

If that's a reliable check for MTU than the problem should not be MTU
related. Is there any other way to check MTU problems by using `ping'?

Thanks,

Volker

On 2005-10-22 20:16, Michael VInce wrote:

Try sending different sized pings or other packet size control utils to
really make sure its not MTU related.
Maybe there is an upstream router thats blocking ICMP fragment packets,
have you ever seen them? try forcing the creation of some.

Mike

Volker wrote:

Still having the same problem with an IPSec tunnel between FreeBSD 5.4R
hosts.

Problem description:
scp session tries to transfer a large file through an IPSec tunnel. The
file is being transmitted but scp says 'stalled' after 56K (49152 bytes
file size). The IPSec tunnel itself is still up even after the scp
abort. Other tcp sessions break, too when sending too much traffic
through the tunnel.

I've taken a closer look to it and tried to get something useful out of
the tcpdump but I'm unable to see any errors or I'm misinterpreting
something.

The connection looks like:

extIP: A.B.C.D
extIP: E.F.G.H
host A ------------------ (internet) ------------------ host B
tunnelIP: 10.128.1.6                                           tunnelIP:
10.128.6.1

host A just has an external interface (em1) connected to a leased line
with a fixed IP address (IP-addr A.B.C.D).
host B has an S-DSL connection at xl0, PPPoE at ng0 (IP-addr. E.F.G.H).

Both hosts are using gif for the IPSec tunnel.

The routing tables (netstat -rnWf inet) are looking good and IMHO the
MTU is fine.

host A:
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
     options=b<RXCSUM,TXCSUM,VLAN_MTU>
     inet A.B.C.D netmask 0xfffffff8 broadcast A.B.C.z
     ether 00:c0:9f:46:ec:c7
     media: Ethernet autoselect (100baseTX <full-duplex>)
     status: active
gif6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
     tunnel inet A.B.C.D --> E.F.G.H
     inet 10.128.1.6 --> 10.128.6.1 netmask 0xffffffff
     inet6 fe80::2c0:9fff:fe46:ecc6%gif6 prefixlen 64 scopeid 0x4

Routing tables (shortened)
Destination        Gateway            Flags    Refs      Use    Mtu
Netif Expire
default            A.B.C.x      UGS         2   516686   1500      em1
10.128.1.6         127.0.0.1          UH          0       14
16384      lo0
10.128.6.1         10.128.1.6         UH          0     6017
1280     gif6
127.0.0.1          127.0.0.1          UH          0    31633
16384      lo0
A.B.C.x/29   link#2             UC          0        0   1500      em1
A.B.C.D      00:c0:9f:46:ec:c7  UHLW        0      112   1500      lo0

On host B the interfaces and routing tables are looking like:
xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
     options=8<VLAN_MTU>
     inet 0.0.0.0 netmask 0xff000000 broadcast 0.255.255.255
     inet6 fe80::260:8ff:fe6c:e73c%xl0 prefixlen 64 scopeid 0x1
     ether 00:60:08:6c:e7:3c
     media: Ethernet 10baseT/UTP (10baseT/UTP <half-duplex>)
     status: active
gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
     tunnel inet E.F.G.H --> A.B.C.D
     inet6 fe80::260:8ff:fe6c:e73c%gif1 prefixlen 64 scopeid 0x4
     inet 10.128.6.1 --> 10.128.1.6 netmask 0xffffffff
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> mtu 1456
     inet E.F.G.H --> 217.5.98.186 netmask 0xffffffff

Routing tables (shortened)
Destination        Gateway            Flags    Refs      Use    Mtu
Netif Expire
0                  link#1             UC          0        0   1500
xl0 =>
default            217.5.98.186       UGS         1    38474
1456      ng0
10.128.1.6         10.128.6.1         UH          4     2196
1280     gif1
127.0.0.1          127.0.0.1          UH          0    80424
16384      lo0
217.5.98.186       E.F.G.H       UH          1        0   1456      ng0
E.F.G.H       lo0                UHS         0        0  16384      lo0

While trying to fetch a file by scp on host A (receiver) from host B
(sender), I captured the following tcpdump on host B:

tcpdump -netttvvi gif1:

000023 AF 2 1280: IP (tos 0x8, ttl  64, id 13202, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
43864:45092(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567
565002838>
000207 AF 2 1280: IP (tos 0x8, ttl  64, id 52187, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567
565002838>
000220 AF 2 1280: IP (tos 0x8, ttl  64, id 33774, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 481770568
565002838>
003524 AF 2 52: IP (tos 0x8, ttl  64, id 42063, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 38952 win 33156 <nop,nop,timestamp 565002844
481770524> 000024 AF 2 1280: IP (tos 0x8, ttl  64, id 48541, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 481770571
565002844>
011203 AF 2 52: IP (tos 0x8, ttl  64, id 60517, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 41408 win 32542 <nop,nop,timestamp 565002855
481770530> 000058 AF 2 1280: IP (tos 0x8, ttl  64, id 15798, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 481770582
565002855>
000246 AF 2 1280: IP (tos 0x8, ttl  64, id 31721, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 481770583
565002855>
005147 AF 2 52: IP (tos 0x8, ttl  64, id 22347, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 42636 win 33156 <nop,nop,timestamp 565002861
481770542> 000024 AF 2 1280: IP (tos 0x8, ttl  64, id 61057, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 481770588
565002861>
020769 AF 2 52: IP (tos 0x8, ttl  64, id 27692, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 565002881
481770547> 000027 AF 2 1280: IP (tos 0x8, ttl  64, id 64167, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609
565002881>
000209 AF 2 1280: IP (tos 0x8, ttl  64, id 45457, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609
565002881>
005260 AF 2 52: IP (tos 0x8, ttl  64, id 53832, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 565002887
481770567> 000024 AF 2 1280: IP (tos 0x8, ttl  64, id 3515, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 481770614
565002887>
011020 AF 2 52: IP (tos 0x8, ttl  64, id 11608, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 565002898
481770568> 000026 AF 2 1280: IP (tos 0x8, ttl  64, id 5848, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625
565002898>
000211 AF 2 1280: IP (tos 0x8, ttl  64, id 39892, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625
565002898>
005641 AF 2 52: IP (tos 0x8, ttl  64, id 7943, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 565002904
481770582> 000024 AF 2 1280: IP (tos 0x8, ttl  64, id 8678, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 481770631
565002904>
011072 AF 2 52: IP (tos 0x8, ttl  64, id 38257, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 565002915
481770583> 000025 AF 2 1280: IP (tos 0x8, ttl  64, id 12255, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642
565002915>
000209 AF 2 1280: IP (tos 0x8, ttl  64, id 46257, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642
565002915>
000222 AF 2 1280: IP (tos 0x8, ttl  64, id 4093, offset 0, flags
[none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 481770643
565002915>
007065 AF 2 52: IP (tos 0x8, ttl  64, id 18720, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 565002922
481770609> 000025 AF 2 1280: IP (tos 0x8, ttl  64, id 38378, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 481770650
565002922>
011034 AF 2 52: IP (tos 0x8, ttl  64, id 18718, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 565002934
481770609> 000024 AF 2 1280: IP (tos 0x8, ttl  64, id 8148, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: .
64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 481770661
565002934>
005991 AF 2 52: IP (tos 0x8, ttl  64, id 62285, offset 0, flags
[none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok]
1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 565002939
481770625> 010726 AF 2 52: IP (tos 0x8, ttl  64, id 1549, offset 0,
flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 565002950
481770625> 005670 AF 2 52: IP (tos 0x8, ttl  64, id 61504, offset 0,
flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 565002956
481770642> 011260 AF 2 52: IP (tos 0x8, ttl  64, id 32633, offset 0,
flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 565002967
481770642> 005510 AF 2 52: IP (tos 0x8, ttl  64, id 54614, offset 0,
flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 565002973
481770650> 104909 AF 2 52: IP (tos 0x8, ttl  64, id 50471, offset 0,
flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum
ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 565003078
481770661>

tcpdump -netttvvi ng0 host A.B.C.D:

000227 AF 2 1352: IP (tos 0x8, ttl  64, id 25895, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10b)
011042 AF 2 128: IP (tos 0x8, ttl  61, id 5786, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf0)
000226 AF 2 1352: IP (tos 0x8, ttl  64, id 36701, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10c)
000216 AF 2 1352: IP (tos 0x8, ttl  64, id 8789, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10d)
004853 AF 2 128: IP (tos 0x8, ttl  61, id 17128, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf1)
000227 AF 2 1352: IP (tos 0x8, ttl  64, id 34888, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10e)
018747 AF 2 128: IP (tos 0x8, ttl  61, id 14828, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf2)
000248 AF 2 1352: IP (tos 0x8, ttl  64, id 34356, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10f)
000223 AF 2 1352: IP (tos 0x8, ttl  64, id 34151, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x110)
005030 AF 2 128: IP (tos 0x8, ttl  61, id 45476, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf3)
000228 AF 2 1352: IP (tos 0x8, ttl  64, id 39765, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x111)
011247 AF 2 128: IP (tos 0x8, ttl  61, id 63692, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf4)
000226 AF 2 1352: IP (tos 0x8, ttl  64, id 29240, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x112)
000222 AF 2 1352: IP (tos 0x8, ttl  64, id 43306, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x113)
005663 AF 2 128: IP (tos 0x8, ttl  61, id 32980, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf5)
000228 AF 2 1352: IP (tos 0x8, ttl  64, id 56920, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x114)
010190 AF 2 128: IP (tos 0x8, ttl  61, id 3206, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf6)
000227 AF 2 1352: IP (tos 0x8, ttl  64, id 4655, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x115)
000215 AF 2 1352: IP (tos 0x8, ttl  64, id 62740, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x116)
000203 AF 2 1352: IP (tos 0x8, ttl  64, id 35642, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x117)
006875 AF 2 128: IP (tos 0x8, ttl  61, id 37801, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf7)
000234 AF 2 1352: IP (tos 0x8, ttl  64, id 41803, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x118)
010651 AF 2 128: IP (tos 0x8, ttl  61, id 54256, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf8)
000235 AF 2 1352: IP (tos 0x8, ttl  64, id 30732, offset 0, flags
[none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x119)
007913 AF 2 128: IP (tos 0x8, ttl  61, id 7647, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf9)
011166 AF 2 128: IP (tos 0x8, ttl  61, id 58037, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfa)
005483 AF 2 128: IP (tos 0x8, ttl  61, id 65275, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfb)
011250 AF 2 128: IP (tos 0x8, ttl  61, id 47289, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfc)
005505 AF 2 128: IP (tos 0x8, ttl  61, id 203, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfd)
104747 AF 2 128: IP (tos 0x8, ttl  61, id 45263, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfe)
8. 338674 AF 2 128: IP (tos 0x8, ttl  61, id 36351, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xff)
319992 AF 2 128: IP (tos 0x8, ttl  61, id 18085, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x100)
441837 AF 2 128: IP (tos 0x8, ttl  61, id 58323, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x101)
684077 AF 2 128: IP (tos 0x8, ttl  61, id 35487, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x102)
1. 167602 AF 2 128: IP (tos 0x8, ttl  61, id 34442, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x103)
2. 136032 AF 2 128: IP (tos 0x8, ttl  61, id 8345, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x104)
2. 984665 AF 2 128: IP (tos 0x8, ttl  61, id 35456, offset 0, flags
[none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x105)

From what I'm seeing host B just stops sending without any reason. At


least I don't see any fragmented packets. The only thing I've seen is
some packets doesn't get ack'ed by the receiver.

These packets never get ack'ed:
46320:47548(1228)
50004:51232(1228)
53688:54916(1228)
57372:58600(1228)
61056:62284(1228)

On host A I dumped the following:

tcpdump -netttvvi gif6

1129985378.941282 AF 2 52: IP (tos 0x8, ttl  64, id 41637, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 574090240
490857876>
1129985378.952628 AF 2 1280: IP (tos 0x8, ttl  64, id 14004, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901
574090210>
1129985378.952657 AF 2 52: IP (tos 0x8, ttl  64, id 23243, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 574090251
490857901>
1129985378.958250 AF 2 1280: IP (tos 0x8, ttl  64, id 4306, offset 0,
flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901
574090210>
1129985378.971118 AF 2 1280: IP (tos 0x8, ttl  64, id 33534, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 490857920
574090229>
1129985378.971137 AF 2 52: IP (tos 0x8, ttl  64, id 60095, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 574090270
490857901>
1129985378.982488 AF 2 1280: IP (tos 0x8, ttl  64, id 11459, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931
574090240>
1129985378.982516 AF 2 52: IP (tos 0x8, ttl  64, id 33184, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 574090281
490857931>
1129985378.987989 AF 2 1280: IP (tos 0x8, ttl  64, id 54180, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931
574090240>
1129985378.994231 AF 2 1280: IP (tos 0x8, ttl  64, id 24535, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 490857942
574090251>
1129985378.994250 AF 2 52: IP (tos 0x8, ttl  64, id 30647, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 574090293
490857931>
1129985379.012101 AF 2 1280: IP (tos 0x8, ttl  64, id 61397, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 490857960
574090270>
1129985379.012132 AF 2 52: IP (tos 0x8, ttl  64, id 60550, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 574090311
490857960>
1129985379.017754 AF 2 1280: IP (tos 0x8, ttl  64, id 28408, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 490857961
574090270>
1129985379.023720 AF 2 1280: IP (tos 0x8, ttl  64, id 27558, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 490857972
574090281>
1129985379.023741 AF 2 52: IP (tos 0x8, ttl  64, id 21502, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 574090322
490857961>
1129985379.035333 AF 2 1280: IP (tos 0x8, ttl  64, id 18885, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
574090293>
1129985379.035362 AF 2 52: IP (tos 0x8, ttl  64, id 59875, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 574090334
490857984>
1129985379.040830 AF 2 1280: IP (tos 0x8, ttl  64, id 37252, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
574090293>
1129985379.046576 AF 2 1280: IP (tos 0x8, ttl  64, id 18349, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984
574090293>
1129985379.046595 AF 2 52: IP (tos 0x8, ttl  64, id 43697, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 574090345
490857984>
1129985379.064961 AF 2 1280: IP (tos 0x8, ttl  64, id 38300, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013
574090322>
1129985379.064993 AF 2 52: IP (tos 0x8, ttl  64, id 47539, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 574090364
490858013>
1129985379.070688 AF 2 1280: IP (tos 0x8, ttl  64, id 30345, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013
574090322>
1129985379.076184 AF 2 1280: IP (tos 0x8, ttl  64, id 37536, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 490858014
574090322>
1129985379.076202 AF 2 52: IP (tos 0x8, ttl  64, id 34201, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 574090375
490858013>
1129985379.081680 AF 2 1280: IP (tos 0x8, ttl  64, id 20637, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 490858025
574090334>
1129985379.081709 AF 2 52: IP (tos 0x8, ttl  64, id 59866, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 574090380
490858025>
1129985379.087678 AF 2 1280: IP (tos 0x8, ttl  64, id 35213, offset
0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: .
64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 490858036
574090345>
1129985379.186906 AF 2 52: IP (tos 0x8, ttl  64, id 2465, offset 0,
flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp
sum ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 574090486
490858036>

tcpdump -netttvvi em1 host E.F.G.H

1129985379.064825 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
IPv4 (0x0800), length 1366: IP (tos 0x8, ttl  61, id 45003, offset 0,
flags [none], length: 1352) E.F.G.H > A.B.C.D:
ESP(spi=0x0e0dffaa,seq=0x3e)
1129985379.065024 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
IPv4 (0x0800), length 142: IP (tos 0x8, ttl  64, id 1195, offset 0,
flags [none], length: 128) A.B.C.D > E.F.G.H:
ESP(spi=0x029a41b4,seq=0x2f)
1129985379.070572 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
IPv4 (0x0800), length 1366: IP (tos 0x8, ttl  61, id 36820, offset 0,
flags [none], length: 1352) E.F.G.H > A.B.C.D:
ESP(spi=0x0e0dffaa,seq=0x3f)
1129985379.076069 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
IPv4 (0x0800), length 1366: IP (tos 0x8, ttl  61, id 44971, offset 0,
flags [none], length: 1352) E.F.G.H > A.B.C.D:
ESP(spi=0x0e0dffaa,seq=0x40)
1129985379.076233 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
IPv4 (0x0800), length 142: IP (tos 0x8, ttl  64, id 56964, offset 0,
flags [none], length: 128) A.B.C.D > E.F.G.H:
ESP(spi=0x029a41b4,seq=0x30)
1129985379.081565 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
IPv4 (0x0800), length 1366: IP (tos 0x8, ttl  61, id 24742, offset 0,
flags [none], length: 1352) E.F.G.H > A.B.C.D:
ESP(spi=0x0e0dffaa,seq=0x41)
1129985379.081741 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
IPv4 (0x0800), length 142: IP (tos 0x8, ttl  64, id 9390, offset 0,
flags [none], length: 128) A.B.C.D > E.F.G.H:
ESP(spi=0x029a41b4,seq=0x31)
1129985379.087562 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype
IPv4 (0x0800), length 1366: IP (tos 0x8, ttl  61, id 48065, offset 0,
flags [none], length: 1352) E.F.G.H > A.B.C.D:
ESP(spi=0x0e0dffaa,seq=0x42)
1129985379.186945 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype
IPv4 (0x0800), length 142: IP (tos 0x8, ttl  64, id 36315, offset 0,
flags [none], length: 128) A.B.C.D > E.F.G.H:
ESP(spi=0x029a41b4,seq=0x32)

If I'm not misleaded, this also doesn't show any errors except the
missing ack's. host B just stops sending. If there's an ack missing,
doesn't have the sending host to just repeat the un-ack'ed packet?

The IPSec tunnel does not die. Even shortly after the (scp) transfer
stalls the tunnel itself is still usable (for small amounts of data). To
make it more worse, when disabling pf at the senders side, the transfer
works. I've tripple checked pflog for denied packets on both sides but
pf didn't filter any packets out.

When disabling the IPSec rules using `setkey -F; setkey -FP' on the
tunnel for a moment, the scp transfer does not stall. So it's not a gif
issue.

It doesn't seem to be an MTU issue (pf has also the rule 'scrub in/out
all no-df'), but what kind of issue is that?? Has anybody ever
experienced similar things? Or am I misinterpreting the tcpdump output?


Any help and hint is appreciated! Without an error message I'm lost.

Volker

_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: IPSec tcp session stalling

Reply via email to