[mp2] Problem with forking cleanup in high request rate server

[Scot Martin]

mp1.99.10-dev
Apache 2.0.47
Solaris 8

I'm building a single-purpose, high-performance, multi-threaded  application server using the above and have run into problems with forking and cleanup of old httpd instances.  Yes, I have to fork and exec as I have to call some legacy C code and then set up comms between the exec'd process and Apache.


My first problem comes when loading the server heavily I occasionally see an instance that forked properly, but the child never wakes up and runs.  I only see this behavior when pushing 10 or so requests per second at the server.  There's probably 3 per minute of these so it's a real needle in a haystack.
Any thoughts on how to prevent this and/or kill the errant kid?


The second problem is after a half hour or so of slamming requests at an Apache instance, the kid hangs.  The parent proc does the right thing and starts another kid but since the first kid is hung, my reaper doesn't do its job and I end up with a bunch of zombies until I slay the kid.  I've tried setting
MaxRequestsPerChild to various values but that doesn't help - in fact, it doesn't seem to do anything.  Am I missing something? 

Lastly, cleanup_for_exec() doesn't seem to work - with tracing on I can see that we get into APR::Pool but the return status is always zero.  I've tried $r->APR::Pool::cleanup_for_exec() and a bald &APR::Pool::cleanup_for_exec().

Here's a code snippet:

        if(!(defined ($pid = fork())))
        {

                return(&RELPERL::appmgr_env::ivruapperr "APP_FORK_ERR","error forking: $!"));
        }

        if($pid)
        {
#               print STDERR ("Parent pid $$ finished, kid's pid: $pid\n");
        }
        else
        {
                if(!(&APR::Pool::cleanup_for_exec()))
                {
                        print STDERR ("cleanup_for_exec failed: $!\n");
                }
                # Close filehandles inherited from parent
                close($tmpserver) or &crit_exit();
                close(STDIN) or &crit_exit();
               .
       .
  .
  exec ...
        }

What am I doing wrong here?


By the way, I've seen a few people ask about how to do zombie cleanup.  I tried all sorts of ways and the only way I found to work on Sol 8 was this:

use POSIX 'WNOHANG';
use POSIX ":sys_wait_h";
my($kid);
do {
      $kid=waitpid(-1,WNOHANG);
} until $kid <= 0;

if(!(defined ($pid = fork())))
{
...


Thanks,
Scot