On 06/10/10 21:52, Nobody wrote: > Spawning child processes to perform tasks > which can easily be performed in Python is inefficient
Not necessarily so, recently I wrote a script which takes a blink of an eye when I pipe through cat/grep to prefilter the lines before doing further complex filtering in python; however when I eliminated the cat/grep subprocess and rewrite it in pure python, what was done in a blink of an eye turns into ~8 seconds (not much to fetter around, but it shows that using subprocess can be faster). I eventually optimized a couple of things and reduced it to ~1.5 seconds, up to which, I stopped since to go even faster would require reading by larger chunks, something which I don't really want to do. The task was to take a directory of ~10 files, each containing thousands of short lines (~5-10 chars per line on average) and count the number of lines which match a certain criteria, a very typical script job, however the overhead of reading the files line-by-line in pure python can be straining (you can read in larger chunks, but that's not the point, eliminating grep may not come for free). -- http://mail.python.org/mailman/listinfo/python-list
