Re: pagecrawling websites with Python

2005-04-05 Thread writeson
Swaroop, Thanks for the reply, I'll take a look at HarvestMan and see if we can use it directly, or get some ideas from the source code. :) Doug -- http://mail.python.org/mailman/listinfo/python-list

Re: pagecrawling websites with Python

2005-04-02 Thread Swaroop C H
On 1 Apr 2005 11:58:11 -0800, writeson <[EMAIL PROTECTED]> wrote: > We've got an application we wrote in Python called pagecrawler that > Does anyone have any insight if this is a reasonable approach to build web > pages, > or if we should look at another design? I don't have an answer to your p

pagecrawling websites with Python

2005-04-01 Thread writeson
Hi all, We've got an application we wrote in Python called pagecrawler that generates a list of URL's based on sql queries. It then runs through this list of URL's 'browsing' one of our staging servers for all those URL's. We do this to build the site dynamically, but each page generated by the UR