For such tasks my choice would be twisted combined with ampoule.Let's you spread out work to whatever amount of processes you desire, maxing out whatever iron you're sitting on..
HTH, Werner http://twistedmatrix.com/trac/ https://launchpad.net/ampoule On 29.05.2012 16:43, Jabba Laci wrote:
Hehe, I just asked this question a few days ago but I didn't become much cleverer: http://www.gossamer-threads.com/lists/python/python/985701 Best, Laszlo On Thu, May 10, 2012 at 2:14 PM, Jabba Laci<jabba.l...@gmail.com> wrote:Hi, I would like to do some parallel programming with Python but I don't know how to start. There are several ways to go but I don't know what the differences are between them: threads, multiprocessing, gevent, etc. I want to use a single machine with several cores. I want to solve problems like this: iterate over a loop (with millions of steps) and do some work at each step. The steps are independent, so here I would like to process several steps in parallel. I want to store the results in a global list (which should be "synchronised"). Typical use case: crawl webpages, extract images and collect the images in a list. What's the best way? Thanks, Laszlo
<<attachment: werner.vcf>>
-- http://mail.python.org/mailman/listinfo/python-list
