after extensive profiling I found out that the way that os.walk() is implemented it calls os.stat() on the dirs and files multiple times and that is where all the time is going.
os.walk() is pretty simple, you could copy it and make your own version that calls os.stat() just once for each item. The dirnames and filenames lists it yields could be lists of (name, os.stat(path)) tuples so you would have the sizes available.
Kent -- http://mail.python.org/mailman/listinfo/python-list
