web crawler in python

[my name]

I'm currently planning on writing a web crawler in python but have a
question as far as how I should design it. My goal is speed and maximum
efficient use of the hardware\bandwidth I have available.

As of now I have a Dual 2.4ghz xeon box, 4gb ram, 500gb sata and a 20mbps
bandwidth cap (for now) . Running FreeBSD.

What would be the best way to design the crawler? Using the thread module?
Would I be able to max out this connection with the hardware listed above
using python threads?

Thank you kindly.
-- 
http://mail.python.org/mailman/listinfo/python-list

FOREX : Foreign Exchange Market : FX : Currency Market : Earn Money Quickly.

[MY NAME IS MY NAME]

FOREX : Foreign Exchange Market : FX : Currency Market : Earn Money
Quickly.

Simple Optimized Tips and Tricks, Basics of FOREX to be a king of
FOREX in one day.

Check out these exclusive never seen tips for free at

http://forex-fx-currencymarket.blogspot.com/2010/02/forex-market-size-and-liquidity-showing.html


Contents Overview :

FOREX : Market Size and Liquidity Showing Top 10 Currency Traders
FOREX : Speculation of Currency
FOREX : Instruments Financially including Transactions
FOREX : FX Rates Determinants as per Government
FOREX : Trading Characteristics and Trading Centers
FOREX : Market Participants with Access Levels
-- 
http://mail.python.org/mailman/listinfo/python-list