[EMAIL PROTECTED] (Scott Kveton) wrote in message news:<[EMAIL PROTECTED]>... > Hello all, > > I'm working on a problem that seems like it should work and am looking for > some help knocking out any obvious problems. > > We currently have several DSL lines that are used for crawling websites. > Bandwidth as it is is pretty cheap via DSL (we have 6 7Mbit lines for much > less than an ~equivalent T-3). Right now we use a single RedHat machine > that has the DSL lines attached to it and we use iproute to direct the > "networks" of traffic to each DSL line.
Hi Scott, So what is this crawler doing then? I noticed you hitting my site the other day, and didn't like the look of someone downloading everysingle one of my images. What's it all for? Thanks, -James.
