Hello Angus: > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On > Behalf Of Angus McIntyre > Sent: Wednesday, July 13, 2005 6:48 AM > To: [email protected] > Subject: fork() and ithreads in AS Perl 5.8.6 > > > I'm writing a Perl program to run on Windows platforms (Win2K and > WinXP). whose job is to manage concurrent downloads using libwww-perl > (LWP). The basic model is that a manager process checks continuously > against a task list (actually a directory containing task files) and > launches a sub-process or thread to handle each download. Because the > downloads are expected to run concurrently, the manager doesn't wait > for the sub-processes to complete. Not sure if fork will do it for you on Win32...threads on Win32 also leak handles if you used any sort of shared hasshes or arrays.
> > My first attempt used fork() to launch the sub-processes, and each > sub-process would then call exit(0) when complete. However, it > appears that exit(0) doesn't release the thread handles: the manager > would fork 64 sub-processes successfully, and then die with 'resource > temporarily unavailable' on the 65th. Note that this wasn't a case of > too many concurrent processes: most of the sub-processes would > already have exited, so there were never more than about 20 processes > active simultaneously. Interesting! Similar bug to the threads functionality? > I then rewrote the code using threads instead. Each thread was > created with new() and then immediately detached using detach(). The > manager doesn't need to get a result back from the thread. > > This performed even worse: after a random number of thread creations > - between about 40 and 60 - Perl would simply exit, and Windows would > dump an exception log that indicated thread-related problems. Again, > the figure of 40-60 refers not to concurrent threads, but to total > threads created. Leaking handles I'm afraid, the bug has been in bugzilla since...I'm not sure when I put it in there. > > Has anyone had any experience in writing multi-threaded processes on > Windows using libwww-perl? Is it possible to do this stably, or is > multi-threading in Windows Perl not yet mature enough? Or, > alternatively, am I making some fundamental error in the way I'm > going about this? > > Thanks in advance for any suggestions, I use the perl threading on Win32 as I have to contact 27 remote server modules ( actually MODBUS devices) every minute and since the socket timeouts in perl do not work on Win32 nor do non-blocking handles, each attempt to connect has to be in its own detached thread so that if the target device isn't there, the thread and can timeout(some wort of windows default, ~20 seconds or so) and die at its leisure... this works simple enough. The problem is, I'm populating some shared hashes with the connection status of each connection attempt and a shared hash to hold the retrieved data....here is where the problem occurs. For every thread launched, perl leaves 2 handles open after that thread dies. I think the max number of handles any app is allowed to have open is 128,000 or so but to get around this, I use task scheduler to kill my app every 5 hours and 58 minutes and then restart it every 6 hours(its not running for 2 minutes in other words). I have to do this till the leaking handles bug gets fixed. Since final results are stored to a file, the hack work around works for me. Cheers, john > > Angus > _______________________________________________ > Perl-Win32-Users mailing list > [email protected] > To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs > _______________________________________________ Perl-Win32-Users mailing list [email protected] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
