Re: FreeBSD 9.0 generates incorrect SEC/ACK numbers under load

Andre Oppermann Fri, 30 Mar 2012 08:13:37 -0700

On 30.03.2012 15:04, Sergey Smitienko wrote:

Here you go, two sessions, one with win set in Syn/Ack packet and other
with separate "windows open" Ack packet.


Thanks for the tcpdumps.  The window update issue seems to be separate
from the seq#ack# problem.

Why do set the recvspace to the very low value of 8192?

Do you have any special settings on this system?  Like transparent
proxying, NAT and so on?

Could you please try the patch after my signature to get a first grip
on the window update problem.

--
Andre

$ svn diff netinet/tcp_syncache.c
Index: netinet/tcp_syncache.c
===================================================================
--- netinet/tcp_syncache.c      (revision 233227)
+++ netinet/tcp_syncache.c      (working copy)
@@ -1080,6 +1080,13 @@
        sb_hiwat = so->so_rcv.sb_hiwat;
        ltflags = (tp->t_flags & (TF_NOOPT | TF_SIGNATURE));

+       if (win != sb_hiwat &&
+           (s = tcp_log_addrs(inc, th, NULL, NULL))) {
+               log(LOG_DEBUG, "%s; %s: win != sb_hiwat\n",
+                   s, __func__);
+               free(s, M_TCPLOG);
+       }
+
        /* By the time we drop the lock these should no longer be used. */
        so = NULL;
        tp = NULL;

16:59:47.629750 IP (tos 0x0, ttl 123, id 55648, offset 0, flags [DF],
proto TCP (6), length 48)
     195.64.148.12.61153>  193.178.147.113.80: Flags [S], cksum 0x5721
(correct), seq 770400880, win 65535, options [mss 1460,nop,nop,sackOK],
length 0
16:59:47.629774 IP (tos 0x0, ttl 254, id 43755, offset 0, flags [DF],
proto TCP (6), length 48)
     193.178.147.113.80>  195.64.148.12.61153: Flags [S.], cksum 0xeaa3
(correct), seq 2323563246, ack 770400881, win 8192, options [mss
1460,sackOK,eol], length 0
16:59:47.631873 IP (tos 0x0, ttl 123, id 40733, offset 0, flags [DF],
proto TCP (6), length 40)
     195.64.148.12.61153>  193.178.147.113.80: Flags [.], cksum 0x3667
(correct), ack 2323563247, win 65535, length 0
16:59:47.633613 IP (tos 0x0, ttl 123, id 36942, offset 0, flags [DF],
proto TCP (6), length 840)
     195.64.148.12.61153>  193.178.147.113.80: Flags [P.], cksum 0xcfb6
(correct), seq 770400881:770401681, ack 2323563247, win 65535, length 800
16:59:47.633710 IP (tos 0x0, ttl 254, id 22412, offset 0, flags [DF],
proto TCP (6), length 1500)
     193.178.147.113.80>  195.64.148.12.61153: Flags [.], seq
2323563247:2323564707, ack 770401681, win 8760, length 1460
16:59:47.633721 IP (tos 0x0, ttl 254, id 22395, offset 0, flags [DF],
proto TCP (6), length 340)
     193.178.147.113.80>  195.64.148.12.61153: Flags [P.], cksum 0x8323
(correct), seq 2323564707:2323565007, ack 770401681, win 8760, length 300
16:59:47.633745 IP (tos 0x0, ttl 254, id 17184, offset 0, flags [DF],
proto TCP (6), length 40)
     193.178.147.113.80>  195.64.148.12.61153: Flags [F.], cksum 0x0a2e
(correct), seq 2323565007, ack 770401681, win 8760, length 0
16:59:47.636215 IP (tos 0x0, ttl 123, id 65415, offset 0, flags [DF],
proto TCP (6), length 40)
     195.64.148.12.61153>  193.178.147.113.80: Flags [.], cksum 0x2c67
(correct), ack 2323565007, win 65535, length 0
16:59:47.636607 IP (tos 0x0, ttl 123, id 48103, offset 0, flags [DF],
proto TCP (6), length 40)
     195.64.148.12.61153>  193.178.147.113.80: Flags [.], cksum 0x2c66
(correct), ack 2323565008, win 65535, length 0
16:59:47.636841 IP (tos 0x0, ttl 123, id 39732, offset 0, flags [DF],
proto TCP (6), length 40)
     195.64.148.12.61153>  193.178.147.113.80: Flags [F.], cksum 0x2c65
(correct), seq 770401681, ack 2323565008, win 65535, length 0
16:59:47.636855 IP (tos 0x0, ttl 254, id 37717, offset 0, flags [DF],
proto TCP (6), length 40)
     193.178.147.113.80>  195.64.148.12.61153: Flags [.], cksum 0x0a2e
(correct), ack 770401682, win 8759, length 0
17:01:58.437891 IP (tos 0x0, ttl 121, id 23760, offset 0, flags [DF],
proto TCP (6), length 48)


     92.231.64.37.61153>  193.178.147.113.80: Flags [S], cksum 0x5c46
(correct), seq 3652856772, win 16384, options [mss 1452,nop,nop,sackOK],
length 0
17:01:58.437907 IP (tos 0x10, ttl 254, id 61730, offset 0, flags [DF],
proto TCP (6), length 44)
     193.178.147.113.80>  92.231.64.37.61153: Flags [S.], cksum 0x2c06
(correct), seq 3164719252, ack 3652856773, win 0, options [mss 1452],
length 0
17:01:58.514354 IP (tos 0x0, ttl 121, id 23780, offset 0, flags [DF],
proto TCP (6), length 40)
     92.231.64.37.61153>  193.178.147.113.80: Flags [.], cksum 0xffaa
(correct), ack 3164719253, win 17424, length 0
17:01:58.514412 IP (tos 0x10, ttl 254, id 17560, offset 0, flags [DF],
proto TCP (6), length 40)
     193.178.147.113.80>  92.231.64.37.61153: Flags [.], cksum 0x23bb
(correct), ack 3652856773, win 8192, length 0
17:01:58.605052 IP (tos 0x0, ttl 121, id 23789, offset 0, flags [DF],
proto TCP (6), length 690)
     92.231.64.37.61153>  193.178.147.113.80: Flags [P.], cksum 0x6b1c
(correct), seq 3652856773:3652857423, ack 3164719253, win 17424, length 650
17:01:58.605123 IP (tos 0x0, ttl 254, id 54275, offset 0, flags [DF],
proto TCP (6), length 1492)
     193.178.147.113.80>  92.231.64.37.61153: Flags [.], seq
3164719253:3164720705, ack 3652857423, win 8712, length 1452
17:01:58.605142 IP (tos 0x0, ttl 254, id 28400, offset 0, flags [DF],
proto TCP (6), length 346)
     193.178.147.113.80>  92.231.64.37.61153: Flags [P.], cksum 0x6b55
(correct), seq 3164720705:3164721011, ack 3652857423, win 8712, length 306
17:01:58.605162 IP (tos 0x0, ttl 254, id 4658, offset 0, flags [DF],
proto TCP (6), length 40)
     193.178.147.113.80>  92.231.64.37.61153: Flags [F.], cksum 0x184a
(correct), seq 3164721011, ack 3652857423, win 8712, length 0
17:01:58.678888 IP (tos 0x0, ttl 121, id 23803, offset 0, flags [DF],
proto TCP (6), length 40)
     92.231.64.37.61153>  193.178.147.113.80: Flags [.], cksum 0xf642
(correct), ack 3164721011, win 17424, length 0
17:01:58.680737 IP (tos 0x0, ttl 121, id 23804, offset 0, flags [DF],
proto TCP (6), length 40)
     92.231.64.37.61153>  193.178.147.113.80: Flags [.], cksum 0xf641
(correct), ack 3164721012, win 17424, length 0
17:01:58.682290 IP (tos 0x0, ttl 121, id 23806, offset 0, flags [DF],
proto TCP (6), length 40)
     92.231.64.37.61153>  193.178.147.113.80: Flags [F.], cksum 0xf640
(correct), seq 3652857423, ack 3164721012, win 17424, length 0
17:01:58.682314 IP (tos 0x0, ttl 254, id 64325, offset 0, flags [DF],
proto TCP (6), length 40)
     193.178.147.113.80>  92.231.64.37.61153: Flags [.], cksum 0x184a
(correct), ack 3652857424, win 8711, length 0

sysctl net.inet.tcp

net.inet.tcp.rfc1323: 1
net.inet.tcp.mssdflt: 1460
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 16384
net.inet.tcp.recvspace: 8192
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.v6mssdflt: 1220
net.inet.tcp.cc.available: newreno
net.inet.tcp.cc.algorithm: newreno
net.inet.tcp.hostcache.purge: 0
net.inet.tcp.hostcache.prune: 300
net.inet.tcp.hostcache.expire: 3600
net.inet.tcp.hostcache.count: 519
net.inet.tcp.hostcache.bucketlimit: 30
net.inet.tcp.hostcache.hashsize: 512
net.inet.tcp.hostcache.cachelimit: 15360
net.inet.tcp.recvbuf_max: 2097152
net.inet.tcp.recvbuf_inc: 16384
net.inet.tcp.recvbuf_auto: 0
net.inet.tcp.insecure_rst: 0
net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.ecn.enable: 0
net.inet.tcp.abc_l_var: 2
net.inet.tcp.rfc3465: 1
net.inet.tcp.rfc3390: 1
net.inet.tcp.rfc3042: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.blackhole: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.sendbuf_max: 65536
net.inet.tcp.sendbuf_inc: 8192
net.inet.tcp.sendbuf_auto: 0
net.inet.tcp.tso: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.reass.overflows: 0
net.inet.tcp.reass.cursegments: 0
net.inet.tcp.reass.maxsegments: 12852
net.inet.tcp.sack.globalholes: 0
net.inet.tcp.sack.globalmaxholes: 65536
net.inet.tcp.sack.maxholes: 128
net.inet.tcp.sack.enable: 1
net.inet.tcp.soreceive_stream: 0
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.pcbcount: 157
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.tcbhashsize: 8192
net.inet.tcp.log_debug: 0
net.inet.tcp.minmss: 216
net.inet.tcp.syncache.rst_on_sock_fail: 1
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.hashsize: 1024
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.cachelimit: 102400
net.inet.tcp.syncache.bucketlimit: 100
net.inet.tcp.syncookies_only: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.timer_race: 0
net.inet.tcp.per_cpu_timers: 0
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.fast_finwait2_recycle: 0
net.inet.tcp.always_keepalive: 1
net.inet.tcp.rexmit_slop: 200
net.inet.tcp.rexmit_min: 30
net.inet.tcp.msl: 30000
net.inet.tcp.nolocaltimewait: 0
net.inet.tcp.maxtcptw: 102400

_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"


_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: FreeBSD 9.0 generates incorrect SEC/ACK numbers under load

Reply via email to