STINNER Victor added the comment: > Strange, I'm unable to reproduce the result on a different computer: (...)
Ah, the benchmark result change after a reboot. I reran the same benchmark (LTO+PGO on speed-python) after a reboot: --- haypo@speed-python$ python3 -m perf compare_to ~/test/2017-01-13_19-18-default-3486425ed82b.json ~/patchdir/patch.json -G Slower (29): - spectral_norm: 275 ms +- 4 ms -> 301 ms +- 16 ms: 1.09x slower (+9%) - fannkuch: 1.10 sec +- 0.01 sec -> 1.16 sec +- 0.18 sec: 1.06x slower (+6%) - scimark_monte_carlo: 282 ms +- 3 ms -> 298 ms +- 6 ms: 1.06x slower (+6%) - sqlite_synth: 9.39 us +- 0.19 us -> 9.82 us +- 0.15 us: 1.05x slower (+5%) - nqueens: 261 ms +- 2 ms -> 271 ms +- 2 ms: 1.04x slower (+4%) - meteor_contest: 209 ms +- 1 ms -> 217 ms +- 2 ms: 1.04x slower (+4%) - scimark_fft: 684 ms +- 22 ms -> 709 ms +- 9 ms: 1.04x slower (+4%) - pickle_pure_python: 1.35 ms +- 0.01 ms -> 1.39 ms +- 0.02 ms: 1.03x slower (+3%) - xml_etree_iterparse: 227 ms +- 5 ms -> 233 ms +- 6 ms: 1.03x slower (+3%) - sympy_integrate: 49.3 ms +- 0.3 ms -> 50.5 ms +- 0.3 ms: 1.02x slower (+2%) - genshi_text: 90.3 ms +- 1.0 ms -> 92.4 ms +- 1.1 ms: 1.02x slower (+2%) - regex_dna: 283 ms +- 1 ms -> 289 ms +- 1 ms: 1.02x slower (+2%) - telco: 22.3 ms +- 0.5 ms -> 22.6 ms +- 0.3 ms: 1.02x slower (+2%) - chaos: 295 ms +- 3 ms -> 300 ms +- 3 ms: 1.02x slower (+2%) - go: 598 ms +- 5 ms -> 607 ms +- 7 ms: 1.01x slower (+1%) - json_dumps: 30.5 ms +- 0.4 ms -> 30.9 ms +- 0.5 ms: 1.01x slower (+1%) - richards: 183 ms +- 4 ms -> 186 ms +- 5 ms: 1.01x slower (+1%) - deltablue: 17.1 ms +- 0.5 ms -> 17.4 ms +- 0.3 ms: 1.01x slower (+1%) - logging_format: 42.1 us +- 0.6 us -> 42.5 us +- 0.6 us: 1.01x slower (+1%) - sympy_str: 511 ms +- 4 ms -> 515 ms +- 5 ms: 1.01x slower (+1%) - 2to3: 760 ms +- 4 ms -> 766 ms +- 4 ms: 1.01x slower (+1%) - regex_compile: 462 ms +- 4 ms -> 465 ms +- 5 ms: 1.01x slower (+1%) - django_template: 447 ms +- 5 ms -> 450 ms +- 4 ms: 1.01x slower (+1%) - logging_simple: 35.6 us +- 0.5 us -> 35.8 us +- 0.6 us: 1.01x slower (+1%) - hexiom: 23.0 ms +- 0.2 ms -> 23.2 ms +- 0.1 ms: 1.01x slower (+1%) - python_startup: 16.0 ms +- 0.1 ms -> 16.1 ms +- 0.1 ms: 1.00x slower (+0%) - regex_effbot: 5.08 ms +- 0.03 ms -> 5.10 ms +- 0.09 ms: 1.00x slower (+0%) - call_method: 13.8 ms +- 0.1 ms -> 13.9 ms +- 0.2 ms: 1.00x slower (+0%) - python_startup_no_site: 9.36 ms +- 0.04 ms -> 9.37 ms +- 0.07 ms: 1.00x slower (+0%) Faster (19): - scimark_lu: 518 ms +- 18 ms -> 458 ms +- 25 ms: 1.13x faster (-11%) - xml_etree_process: 242 ms +- 4 ms -> 234 ms +- 4 ms: 1.03x faster (-3%) - unpickle: 36.0 us +- 0.5 us -> 35.0 us +- 0.5 us: 1.03x faster (-3%) - chameleon: 30.6 ms +- 0.2 ms -> 29.8 ms +- 0.3 ms: 1.03x faster (-3%) - nbody: 240 ms +- 2 ms -> 234 ms +- 2 ms: 1.02x faster (-2%) - call_method_slots: 13.9 ms +- 0.2 ms -> 13.6 ms +- 0.4 ms: 1.02x faster (-2%) - unpickle_pure_python: 924 us +- 8 us -> 904 us +- 9 us: 1.02x faster (-2%) - xml_etree_generate: 289 ms +- 4 ms -> 283 ms +- 4 ms: 1.02x faster (-2%) - mako: 43.2 ms +- 0.4 ms -> 42.3 ms +- 0.4 ms: 1.02x faster (-2%) - pickle_dict: 65.5 us +- 4.1 us -> 64.4 us +- 0.9 us: 1.02x faster (-2%) - call_simple: 12.8 ms +- 0.4 ms -> 12.6 ms +- 0.4 ms: 1.01x faster (-1%) - xml_etree_parse: 294 ms +- 13 ms -> 291 ms +- 9 ms: 1.01x faster (-1%) - pickle_list: 9.38 us +- 0.35 us -> 9.32 us +- 0.14 us: 1.01x faster (-1%) - unpack_sequence: 118 ns +- 3 ns -> 118 ns +- 1 ns: 1.00x faster (-0%) - sympy_expand: 1.14 sec +- 0.01 sec -> 1.13 sec +- 0.01 sec: 1.00x faster (-0%) - sqlalchemy_declarative: 338 ms +- 3 ms -> 337 ms +- 3 ms: 1.00x faster (-0%) - crypto_pyaes: 236 ms +- 2 ms -> 235 ms +- 1 ms: 1.00x faster (-0%) - pidigits: 321 ms +- 1 ms -> 320 ms +- 1 ms: 1.00x faster (-0%) - regex_v8: 45.6 ms +- 0.7 ms -> 45.6 ms +- 0.3 ms: 1.00x faster (-0%) Benchmark hidden because not significant (16): call_method_unknown, dulwich_log, float, genshi_xml, html5lib, json_loads, logging_silent, pathlib, pickle, raytrace, scimark_sor, scimark_sparse_mat_mult, sqlalchemy_imperative, sympy_sum, tornado_http, unpickle_list --- Lighter verbose with a threshold of 5%: --- haypo@speed-python$ python3 -m perf compare_to ~/test/2017-01-13_19-18-default-3486425ed82b.json ~/patchdir/patch.json -G --min-speed=5 Slower (3): - spectral_norm: 275 ms +- 4 ms -> 301 ms +- 16 ms: 1.09x slower (+9%) - fannkuch: 1.10 sec +- 0.01 sec -> 1.16 sec +- 0.18 sec: 1.06x slower (+6%) - scimark_monte_carlo: 282 ms +- 3 ms -> 298 ms +- 6 ms: 1.06x slower (+6%) Faster (1): - scimark_lu: 518 ms +- 18 ms -> 458 ms +- 25 ms: 1.13x faster (-11%) Benchmark hidden because not significant (60): 2to3, call_method, ... --- spectral_norm and scimark_lu are two unstable benchmarks. Sadly, it seems like the current patch is not really significant on benchmarks. But again, it doesn't optimize __call__() slot yet. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29259> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
