On May 29, 2008, at 3:30 PM, Matthew Dillon wrote:
It is quite possible that the other ends of the connection are
still
live and that the issue could very well be a timeout setting in the
server config file instead of something in the TCP stack.
I think we're onto something here, but for some reason it doesn't make
any sense. I have keepalives turned OFF in Apache:
In the OS I have:
net.inet.tcp.keepidle: 900000
net.inet.tcp.keepintvl: 30000
net.inet.tcp.keepinit: 75000
net.inet.tcp.always_keepalive: 1
I see FIN_WAIT_1 connections collecting like crazy again. I managed
to investigate one that's been hung around for several minutes.
1.1.1.1 = our server ip
2.2.2.2 = remote client ip
[mirror0:/usr/local/etc/apache22] netstat -an | grep 2.2.2.2
tcp4 0 25861 1.1.1.1.80 2.2.2.2.33379 FIN_WAIT_1
When I tcpdump this, I see something sending ack's back and forth
every 60 seconds, but what? Apache? I'm not sure why. I don't see
any timeouts in Apache for ~60 seconds. As you can see, sometimes we
send an ack, but never see a reply. I'm gathering the OS level
keepalives don't come into play because this session is not considered
idle?
0:13:07.640426 IP 1.1.1.1.80 > 2.2.2.2.33379: .
4208136508:4208136509(1) ack 1471446041 win 520 <nop,nop,timestamp
3019088951 5004131>
20:13:07.736505 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5022148 3019088951>
20:14:07.702647 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019148951 5022148>
20:15:07.764920 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019208951 5022148>
20:15:07.860988 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5058183 3019208951>
20:16:07.827262 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019268951 5058183>
20:16:07.923341 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5076200 3019268951>
20:17:07.889690 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019328951 5076200>
20:17:07.984770 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5094217 3019328951>
20:18:07.952167 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019388951 5094217>
20:18:08.048249 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5112234 3019388951>
20:19:08.014715 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019448951 5112234>
20:19:08.110803 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5130252 3019448951>
20:20:08.077321 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019508951 5130252>
20:21:08.139995 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019568951 5130252>
20:21:08.236063 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5166286 3019568951>
20:22:08.202435 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019628951 5166286>
20:22:08.297499 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5184303 3019628951>
20:23:08.264631 IP 1.1.1.1.80 > 2.2.2.2.33379: . 0:1(1) ack 1 win 520
<nop,nop,timestamp 3019688951 5184303>
20:23:08.360700 IP 2.2.2.2.33379 > 1.1.1.1.80: . ack 0 win 0
<nop,nop,timestamp 5202321 3019688951>
I'm finding several of these sessions doing the same exact thing....
--
Robert Blayzor, BOFH
INOC, LLC
[EMAIL PROTECTED]
http://www.inoc.net/~rblayzor/
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
