New submission from STINNER Victor: Attached patch makes different changes to the timeit module:
* Display the average, rather than the minimum, of the timings *and* display the standard deviation. It should help a little bit to get more reproductible results. * Change the default repeat from 3 to 5 to have a better distribution of timings. It makes the timeit CLI 66% slower (ex: 1 second instead of 600 ms). That's the price of stable benchmarks :-) * Don't disable the garbage collector anymore! Disabling the GC is not fair: real applications use it. * autorange: start with 1 loop instead of 10 for slow benchmarks like time.sleep(1) * Display large number of loops as power of 10 for readability, ex: "10^6" instead of "1000000". Also accept "10^6" syntax for the --num parameter. * Add support for "ns" unit: nanoseconds (10^-9 second) I consider that these changes are well contained enough to still be ok for 3.6 beta 2. But I add Ned Deily as CC to double check ;-) This patch is related to my work on Python benchmarks, see: * http://perf.readthedocs.io/en/latest/ * https://github.com/python/performance The perf module runs benchmarks in multiple child processes to test different memory layouts (Linux uses ASRL by default) and different hash functions. It helps to get more stable benchmark results, but it's probably overkill for the tiny timeit module. By the way, the "pyperf timeit" command reuses the timeit module of the stdlib. Note: The timeit module still uses the old getopt module which is very strict. For example "python3 -m timeit pass -v" is not recognized ("-v" is read as a statement part of the benchmark, not as --verbose). But I was too lazy to also modify this part, I may do it later ;-) ---------- components: Benchmarks files: timeit.patch keywords: patch messages: 277142 nosy: brett.cannon, fijall, haypo, ned.deily, pitrou, serhiy.storchaka, yselivanov priority: normal severity: normal status: open title: Enhance the timeit module type: performance versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file44780/timeit.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28240> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
