>>>>> "James" == James Lucero <[EMAIL PROTECTED]> writes:
James> Has anyone successfully used the Parallel::ForkManager
James> to download web pages consistently?
My series of "Link Checkers" in my columns have had that problem as
well. The most recent incarnation does a preforking with a
hand-rolled RPC, which I wrote before seeing Parallel::ForkManager,
and can be read about in
http://www.stonehenge.com/merlyn/LinuxMag/col15.html
and
http://www.stonehenge.com/merlyn/LinuxMag/col16.html
I must say, I'm concerned about this line though:
James> Ultimately I need to download hundreds/thousands of pages more
James> efficiently.
Who's pages? Are you following the robot protocol? Are you building
an index? Do you have permission?
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[EMAIL PROTECTED]> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
