RE: [PHP] Re: Crawlers (was parsing large files - PHP or Perl)

2005-02-17 Thread Warren Vail
Check out PHPdig; http://www.phpdig.net/ Warren > -Original Message- > From: Jamie Alessio [mailto:[EMAIL PROTECTED] > Sent: Thursday, February 17, 2005 9:22 AM > To: John Cage > Cc: php-general@lists.php.net > Subject: [PHP] Re: Crawlers (was parsing large f

[PHP] Re: Crawlers (was parsing large files - PHP or Perl)

2005-02-17 Thread Jamie Alessio
Is there anyone on this list who has written fast and decent crawlers in PHP who would be willing to share their experiences? My first inclination would be to use an existing crawler to grab the pages and store all the files locally (even if only temporarily). Then, you can use PHP to do whatev