As I've mentioned before, we (apache.org) are seeing our qpsmtpd processes
stuck in an infinite loop. We're using qpsmtpd-forkserver.
At Ask's suggestion, I've placed in Carp::Confess lines - but the program
isn't accepting signals when we reach this loop state. At all other times,
it'll respond to SIGUSR2 and print the perl stack trace.
I can't consistently reproduce this; but we get 5-10 perl processes a day
that are eating up all of our CPU.
Any help is appreciated. If you need/want coredumps, let me know. I'm
available on AIM if you want to help me debug this in real-time (email me
for my AIM nick). We have a collection of these processes SIGSTOPed so we
can examine them.
Does anyone think upgrading to perl 5.8.7 help?
Thanks! -- justin
gdb stack trace:
(gdb) where
#0 0x281040d2 in Perl_newSVpvn ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#1 0x280e9526 in Perl_magic_setsig ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#2 0x280e782b in Perl_mg_set ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#3 0x280f241d in Perl_pp_sassign ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#4 0x280f1fad in Perl_runops_standard ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#5 0x280a0c26 in S_run_body ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#6 0x280a08b1 in perl_run ()
from /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
#7 0x08049040 in main ()
pstack output:
57376: /usr/local/bin/perl5.8.6
----------------- thread -1 (running) -----------------
0x281040d2 Perl_newSVpvn (80543b0, 4, bfbfec40, 280e9306, 808cad8, 50) + 1e
0x280e9526 Perl_magic_setsig (808cb5c, 8098b38, bfbfecd8, 280f67ca,
808cad8, 8568044) + 22e
0x280e782b Perl_mg_set (808cb5c, 83e8844, 2, 280f238a, 808ff18, 820c094) +
6f
0x280f241d Perl_pp_sassign (80484b4, bfbfed24, 2804ff2b, 28178564,
bfbfee34, 9) + a1
0x280f1fad Perl_runops_standard (2806a5d8, 28178564, bfbfed88, 280a08b1,
1, 0) + 35
0x280a0c26 S_run_body (1, 0, 28052cdd, 2807a040, 804a150, bfbfed64) + 12e
0x280a08b1 perl_run (804c030, 804a0fc, bfbfedec, 8048fd5, 2806a5d8,
2827251c) + 8d
0x8049040 main (9, bfbfee34, bfbfee5c, 8048eb2, 8049b14, 0) + d8
0x8048ebd _start (bfbfeee4, bfbfeef2, bfbfeef6, bfbfef0b, bfbfef0e,
bfbfef11) + 85
lsof output:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
perl5.8.6 57376 smtpd cwd VDIR 4,14 512 2956294
/usr/home/smtpd/qpsmtpd
perl5.8.6 57376 smtpd rtd VDIR 4,14 512 2 /
perl5.8.6 57376 smtpd txt VREG 4,14 10076 2190650
/usr/local/bin/perl
perl5.8.6 57376 smtpd txt VREG 4,14 142236 455321
/libexec/ld-elf.so.1
perl5.8.6 57376 smtpd txt VREG 4,14 1131877 2482217
/usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
perl5.8.6 57376 smtpd txt VREG 4,14 120004 2841954 /lib/libm.so.3
perl5.8.6 57376 smtpd txt VREG 4,14 28644 2833198
/lib/libcrypt.so.2
perl5.8.6 57376 smtpd txt VREG 4,14 43108 2841956
/lib/libutil.so.4
perl5.8.6 57376 smtpd txt VREG 4,14 889908 2841961 /lib/libc.so.5
perl5.8.6 57376 smtpd txt VREG 4,14 7038 2482042
/usr/local/lib/perl5/5.8.6/mach/auto/Sys/Hostname/Hostname.so
perl5.8.6 57376 smtpd txt VREG 4,14 17360 2481822
/usr/local/lib/perl5/5.8.6/mach/auto/IO/IO.so
perl5.8.6 57376 smtpd txt VREG 4,14 18438 2481805
/usr/local/lib/perl5/5.8.6/mach/auto/Fcntl/Fcntl.so
perl5.8.6 57376 smtpd txt VREG 4,14 112273 2481994
/usr/local/lib/perl5/5.8.6/mach/auto/POSIX/POSIX.so
perl5.8.6 57376 smtpd txt VREG 4,14 7630 2081259 /
(/dev/da0s1a)
perl5.8.6 57376 smtpd txt VREG 4,14 27641 2482012
/usr/local/lib/perl5/5.8.6/mach/auto/Socket/Socket.so
perl5.8.6 57376 smtpd 0r VCHR 2,2 0t0 22 /dev/null
perl5.8.6 57376 smtpd 1u PIPE 0xc3d5622c 0 ->0xc3d56180
perl5.8.6 57376 smtpd 2u PIPE 0xc3d5622c 0 ->0xc3d56180
perl5.8.6 57376 smtpd 3r VREG 4,14 6131 2955766
/usr/home/smtpd/qpsmtpd/qpsmtpd-forkserver
perl5.8.6 57376 smtpd 5u IPv4 0t0 TCP no PCB,
CANTSENDMORE, CANTRCVMORE
