On 05/10/2012 06:46 AM, Devin Jeanpierre wrote: > On Thu, May 10, 2012 at 8:14 AM, Jabba Laci <jabba.l...@gmail.com> wrote: >> What's the best way? > >>From what I've heard, http://scrapy.org/ . It is a single-thread > single-process web crawler that nonetheless can download things > concurrently.
Yes, for i/o bound things, asynchronous (event-driven callbacks, where events are triggered by the data) will usually beat multi-threaded. Sometimes a combination of multi-threaded and asynchronous is necessary (thread pools). Twisted is another asynchronous framework core that is very popular for lots of things, both clients and servers. -- http://mail.python.org/mailman/listinfo/python-list
