On 11 Sep 2007 at 16:09, Chas Owens wrote:
> On 9/11/07, Jenda Krynicky <[EMAIL PROTECTED]> wrote:
> > On 11 Sep 2007 at 15:15, Srinivas wrote:
> > > I want to write a perl script that scrapes various job sites like
> > > monster, dice, career builders etc.
> > >
> > > Given the job id and web site name it should scrape the
> > > information and store in a mySQL database.
> >
> > And are you sure they won't mind? I don't work there anymore, but
> > still ... you should make sure what you plan to do is OK with them.
> snip
>
> The easiest way to do this is to obey their robots.txt file. You can
> learn more about robots.txt here:
> http://www.robotstxt.org/wc/faq.html. Also, be careful, the text you
> are copying is still copyrighted and you cannot republish more than a
> snippet without running into potential legal hazards.
I don't think that's enough. It's one thing to index a site for searching
(think ... Google) and another
to scrape the data and present it elsewhere as yours. The fact that it's OK to
run a script to
download some data doesn't mean all uses of said data are all right.
Jenda
=============== [EMAIL PROTECTED] ==================
Always code as if the guy who ends up maintaining your code will be a
violent psychopath who knows where you live.
-- Rick Osborne, <[EMAIL PROTECTED]>
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
