On 2007-09-13, Amit N <[EMAIL PROTECTED]> wrote: > Hi guys, > > I tend to ramble, and I am afraid none of you busy experts will bother > reading my long post, so I will try to summarize it first:
I haven't read the details, but you seem to aim for a single python program that does 'it'. A single sequential thread is not fast enough, so you want parallel execution (and yes there are a zillion ways of doing that). Why don't you start at the other end? Write a program for each task that you have, then fork/spawn/chain/whatever enough processes at OS level to eat all your data. The OS is usually much better at balancing CPU's. The Python module 'subprocess' would be your friend in that case. In addition, you can run each program independently, which will come in handy one day. Sincerely, Albert -- http://mail.python.org/mailman/listinfo/python-list
