occasional ECONNREFUSED when connect()ing.

Deomid Ryabkov Fri, 18 Feb 2005 05:19:09 -0800

I have a strange case of occasional refused connect()'s.
The system is running 4.10-STABLE.

This was originally reported by one of our developers.
The test script he provided is a simple Perl script that repeatedly fetches an URL
from Apache running on this same server.
In the run of 2000 iterations, there are 5-10 random errors fetching the URL.
After further investigation and digging, I uncovered the true cause of the error:
connect() returning ECONNREFUSED.
To see if this was somehow related to Apache, I wrote my own simple server that
accept()'s a connection, read()'s what supposed to be request and write()'s a stereotypic reply,
thus resembling an HTTP request/reply conversation.
The number of sporadically occuring connect() errors has increased somewhat,
maybe due to quicker turnaround of my stub-server.
The question is - why do connections get refused at all?
I can think of no valid reasons...

here's the output of the client as i see it:

----------------------------- sample output of x.pl -----------------------------
71590, http://localhost:8000/junk, 2000: starting.
59 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
127 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
213 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
259 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
363 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
402 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
416 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
690 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
703 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
754 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
947 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
960 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
997 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
1163 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
1272 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
1433 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
1612 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
1887 500 Can't connect to localhost:8000 (Timeout connect: Connection refused)
----------------------------- cut -----------------------------
(the confusing "Timeout connect:" message is a result of my IO/Socket/INET.pm hacking.
without it, just a confusing "Timeout" was reported. i added printing of actual error string.)

a fragment of output of the same script running under strace:

----------------------------- strace of x.pl -----------------------------
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)
----------------------------- cut -----------------------------

and the source of both server and client used:

----------------------------- accepter.c (server) -----------------------------
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#include <stdio.h>

char *reply = "HTTP/1.0 200 OK\r\nConetnt-type: text/plain\r\n\r\nNothing special.\r\n";
int rlen;

int main(int argc, char **argv) {

       int i, cal;
       int s, ss, n;
       char *buf;
       struct sockaddr_in saddr, caddr;

       memset(&saddr, 0, sizeof saddr);
       saddr.sin_family = AF_INET;
       saddr.sin_addr.s_addr = htonl(INADDR_ANY);
       saddr.sin_port = htons(8000);

ss = socket(PF_INET, SOCK_STREAM, 6);
if (ss < 0) return 1;
if (bind(ss, (struct sockaddr *) &saddr, sizeof saddr) < 0) return 2;
if (listen(ss, 511) < 0) return 3;

       buf = (char *) malloc(4096);
       if (buf == NULL) return 4;

       rlen = strlen(reply);

       fprintf(stderr, "accepting.\n");

       for (cal = sizeof caddr;
            s = accept(ss, (struct sockaddr *) &caddr, &cal);
            cal = sizeof caddr) {

               n = read(s, buf, 4096);
               if (n > 0) {
                       write(s, reply, rlen);
               }
               close(s);
       }

       return (s < 0);
}
----------------------------- cut -----------------------------

----------------------------- x.pl (client) -----------------------------
#!/usr/bin/perl

$| = 1;

use LWP::UserAgent;

$url = 'http://localhost:8000/junk';

my $n = 2000;

print "$$, $url, $n: starting.\n";

for (my $i = 0; $i < $n; $i++) {
       my $ua = LWP::UserAgent->new;
       $ua->timeout(1000);
       $req = HTTP::Request->new("GET" => $url);
       $res = $ua->request($req);

       print $i." ".$res->status_line."\n" if (0 or $res->code == 500);
}

exit();
----------------------------- cut -----------------------------

--
Deomid Ryabkov aka Rojer
[EMAIL PROTECTED]
[EMAIL PROTECTED]
ICQ: 8025844

smime.p7s
Description: S/MIME Cryptographic Signature

occasional ECONNREFUSED when connect()ing.

Reply via email to