I guess that the code
is running in kernel mode. I think this because I can send a KILL signal
to it and the state changes to the following:
What's your operating system and file system? A better file system or
system setting may increase your performance a lot. XFS or ext3 / ext4
with hashed directory index increases the file lookup part from O(n) to
O(1).
FreeBSD 8.1 and UFS2 + soft updates. Dirhash memory already set to a
high value:
vfs.ufs.dirhash_maxmem: 134217728
It's mostly likely a problem with the OS, hardware and/or your
configuration. Python doesn't come with its own stat() implementation.
os.stat() just wraps the libc's stat() function. The heavy lifting is
done inside libc and the kernel.
Great, so the I should ask the BSD list about this.
I'll first try to rewrite the code and replace that some million files
with a very big hash database (gdbm). I guess that will be faster.
Thanks.
Laszlo
--
http://mail.python.org/mailman/listinfo/python-list
