STINNER Victor added the comment:
I also ran the benchmark without cache on disk. It looks like my hard drive is
too slow to see a real speedup of scandir(). The maximum speedup is 2.7 seconds
lesser when testing the "c" scandir (24.089 => 21.374 seconds): "1.1x as fast".
I modified the benchmark to flush all caches, I added the following line into
do_scandir_walk():
os.system("sudo bash -c 'echo 3 > /proc/sys/vm/drop_caches'")
I also commented the first call to do_scandir_walk() ("Priming the system's
cache..."), just to make the benchmark faster.
(See attached clear_system_cache.patch for all changes.)
Result of the modified benchmark without cache:
haypo@smithers$ python3 setup.py build && for scandir in generic python c; do
echo; echo "=== $scandir ==="; PYTHONPATH=build/lib.linux-x86_64-3.3/ python3
benchmark.py /usr/share -c $scandir || break; done
running build
running build_py
running build_ext
=== generic ===
Using very slow generic version of scandir
Comparing against builtin version of os.walk()
Benchmarking walks on /usr/share, repeat 1/3...
Benchmarking walks on /usr/share, repeat 2/3...
Benchmarking walks on /usr/share, repeat 3/3...
os.walk took 24.324s, scandir.walk took 24.215s -- 1.0x as fast
=== python ===
Using slower ctypes version of scandir
Comparing against builtin version of os.walk()
Benchmarking walks on /usr/share, repeat 1/3...
Benchmarking walks on /usr/share, repeat 2/3...
Benchmarking walks on /usr/share, repeat 3/3...
os.walk took 24.089s, scandir.walk took 21.374s -- 1.1x as fast
=== c ===
Using fast C version of scandir
Comparing against builtin version of os.walk()
Benchmarking walks on /usr/share, repeat 1/3...
Benchmarking walks on /usr/share, repeat 2/3...
Benchmarking walks on /usr/share, repeat 3/3...
os.walk took 24.225s, scandir.walk took 21.390s -- 1.1x as fast
----------
Added file: http://bugs.python.org/file36847/clear_system_cache.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue22524>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
