Re: Web scraping...move on if it fails

2008-12-14 Thread Chas. Owens
On Sat, Dec 13, 2008 at 13:26, hotkitty wrote: > HI, > > I have a bunch of news websites that are stored in my mysql db and > each morning I have a script to go to each site and download the top > stories (this is for personal use, not commercial). My problem is that > sometimes www::mechanize wil

Web scraping...move on if it fails

2008-12-14 Thread hotkitty
HI, I have a bunch of news websites that are stored in my mysql db and each morning I have a script to go to each site and download the top stories (this is for personal use, not commercial). My problem is that sometimes www::mechanize will fail to get the website because the server is busy, or fo

Re: Web scraping...move on if it fails

2008-12-14 Thread Filip van der Meeren
I am not familiar with the package, but personally I would place it in a eval block. Filip van der Meeren fi...@code2develop.com http://www.sourceforge.net/projects/perlmanager http://www.sourceforge.net/projects/xlinterpreter On 13 Dec 2008, at 19:26, hotkitty wrote: HI, I have a bunch of

Web scraping...move on if it fails

2008-12-14 Thread hotkitty
HI, I have a bunch of news websites that are stored in my mysql db and each morning I have a script to go to each site and download the top stories (this is for personal use, not commercial). My problem is that sometimes www::mechanize will fail to get the website because the server is busy, or fo