On Jan 29, 7:07 pm, Steven D'Aprano <st...@remove-this- cybersource.com.au> wrote: > On Fri, 29 Jan 2010 14:49:06 -0800, Jonathan Gardner wrote: > > On Jan 28, 3:52 pm, elsa <kerensael...@hotmail.com> wrote: > > >> I've got a problem with my program, in that the code just takes too > >> long to run. Here's what I'm doing. If anyone has any tips, they'd be > >> much appreciated! > > > First of all, don't play with large lists. Large lists have a tendency > > to grow larger over time, until they get absurdly large. > > > If you're dealing with a long list, work with it like you'd work with a > > file. If you need random access, stick it in some form of database, such > > as BDB or PostgreSQL. If you need an index, stick it in some form of DB. > > Eventually, large lists end up like that anyway. Don't fool yourself > > early on---prepare for the worst and solve the problems of tomorrow > > today. > > Talk about premature optimization. The OP probably has a few hundreds of > thousands of items, millions at most, which is trivial on a modern PC. > Your way of thinking is what has led to Firefox using a database to > manage a few thousand bookmarks and cookies, which in turn leads to > consistent data corruption problems. (I've been keeping note, and so far > my installation of Firefox 3 has had database corruption at startup one > time out of five.) >
I think there's a difference between writing your code so that you have the option of using a database, or a flat file, or any other type of serial data source, and actually using a database (or whatever.) I prefer to write my code so that I don't have to re-write it 6 months later. Having worked with datasets that are truly enormous, whenever I see a list that is unbounded (as this appears to be), I freak out because unbounded can be a really, really big number. Might as well write software that works for the truly enormous case and the really large case and be done with it once and for all. -- http://mail.python.org/mailman/listinfo/python-list
