On Mar 19, 10:35 am, Jean-Paul Calderone <exar...@divmod.com> wrote: > On Thu, 19 Mar 2009 09:50:51 -0700, Ryan Rosario <uclamath...@gmail.com> > wrote: > >I have a parser that needs to process 7 million files. After running > >for 2 days, it had only processed 1.5 million. I want this script to > >parse several files at once by using multiple threads: one for each > >file currently being analyzed. > > Threads don't magically make a program faster. In Python in particular, > threads won't do much, if anything, to speed up a CPU bound task. Assuming > you have more than one CPU, Python still limits you to one thread executing > Python bytecode at a time. You could try running multiple processes instead, > if there really is more hardware that's sitting idle with your single > threaded version. > > Jean-Paul
Thank you for your response. I did not realize that. That seems like a huge limitation for such a great language. I will look into forking off processes instead of using threads. R. -- http://mail.python.org/mailman/listinfo/python-list
