On Wed, Aug 22, 2012 at 5:23 PM, Pete O'Connell <pedrooconn...@gmail.com> wrote: > OK maybe I am wrong about it being slow (I thought for loops were > slower than lis comprehensions). But I do know I need it to be as fast > as possible if I need to run it on a thousand files each with hundreds > of thousands of lines
You're quite likely wrong about that. The overall runtime of your code is very likely to end up dominated by I/O with the hard drive, not the microsecond differences in how you process each line. The way to know for sure is to write code that is easy to read and does the correct thing. Then run the code an see if it's too slow for your purposes. If it is, profile the code and see where it's spending all of its time. Only then can you actually optimize the right places. Once you've identified where your code needs to be faster, you're likely to get more mileage out of a clever algorithm change than a micro-optimization to your filtering. Or maybe you'll discover that it really is worthwhile to optimize your filtering, because it's inside a tight loop, and that's where all of your time is spent. There are a lot of tricks you can use to slightly speed up your python code, but none of them are worth doing until you know for sure that you're optimizing in the right place. And if it turns out that the code's runtime is fine as first written, you don't have to waste your time doing lots of work for little gain. -- Jerry _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
