Andre Oppermann wrote:
Mark Hills wrote:
On Mon, 21 Apr 2008, Andre Oppermann wrote:
Mark Hills wrote:
On Sun, 20 Apr 2008, Peter Jeremy wrote:
I can't explain the problem but it definitely looks like a resource
starvation issue within the kernel.
I've traced the source of the ETIMEDOUT within the kernel to
tcp_timer_rexmt() in tcp_timer.c:
if (++tp->t_rxtshift > TCP_MAXRXTSHIFT) {
tp->t_rxtshift = TCP_MAXRXTSHIFT;
tcpstat.tcps_timeoutdrop++;
tp = tcp_drop(tp, tp->t_softerror ?
tp->t_softerror : ETIMEDOUT);
goto out;
}
Yes, this is related to either lack of mbufs to create a segment
or a problem in sending it. That may be full interface queue, a
bandwidth manager (dummynet) or some firewall internally rejecting
the segment (ipfw, pf). Do you run any firewall in stateful mode?
There's no firewall running.
I'm new to FreeBSD, but it seems to implies that it's reaching a
limit of a number of retransmits of sending ACKs on the TCP
connection receiving the inbound data? But I checked this using
tcpdump on the server and could see no retransmissions.
When you have internal problems the segment never makes it to the
wire and thus you wont see it in tcpdump.
Please report the output of 'netstat -s -p tcp' and 'netstat -m'.
Posted below. You can see it it in there: "131 connections dropped by
rexmit timeout"
As a test, I ran a simulation with the necessary changes to increase
TCP_MAXRXTSHIFT (including adding appropriate entries to
tcp_sync_backoff[] and tcp_backoff[]) and it appeared I was able to
reduce the frequency of the problem occurring, but not to a usable
level.
Possible causes are timers that fire too early. Resource starvation
(you are doing a lot of traffic). Or of course some bug in the code.
As I said in my original email, the data transfer doesn't stop or
splutter, it's simply cut mid-flow. Sounds like something happening
prematurely.
Thanks for the help,
The output doesn't show any obvious problems. I have to write some
debug code to run on your system. I'll do that later today if time
permits. Otherwise tomorrow.
http://people.freebsd.org/~andre/tcp_output-error-log.diff
Please apply this patch and enable the sysctl net.inet.tcp.log_debug=1
and report any output. You likely get some (normal) noise from syncache.
What we are looking for is reports from tcp_output.
--
Andre
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
