[Python-Dev] Add tp_fastcall to PyTypeObject

2017-01-13 Thread Victor Stinner 
Hi,

tl;dr Python 3.7 is going to be faster without breaking backward
compatibility, say hello to the new "tp_fastcall" slot!
=> http://bugs.python.org/issue29259


Python 3.6 got a new "FASTCALL" calling convention which allows to
avoid the creation a temporary tuple to pass positional arguments and
a temporary dictionary to pass keyword arguments. But callable objects
having a __call__() method implemented in Python don't benefit of
FASTCAL yet.

I tried to reuse the tp_call slot with a new flag in tp_flags, but I
had two major blocker issues:

* Deeply break the backward compatibility of the C API: calling
directly tp_call (with tuple/dict) would crash immediately if the
object uses FASTCALL

* Need to duplicate each "tp_call" function to get a new "tp_fastcall"
flavor. It wasn't easy to share the function body.


Good news, I found a new design which don't have any of these issues!
=> http://bugs.python.org/issue29259

I chose to add a new tp_fastcall field to PyTypeObject and use a tiny
wrapper calling tp_fastcall for tp_call, to keep the backward
compatibility.


The goal is to get optimizations "for free" when calling functions.
The best expected speedup on a microbenchmark is around 1.56x faster
(-36%) when calling an object supporting FASTCALL. Example with
property_descr_get() without its "cached args" hack, result without
fastcall ("py34") compared to fastcall ("fastcall_wrapper"):

Median +- std dev: [py34] 75.0 ns +- 1.7 ns -> [fastcall_wrapper] 48.2
ns +- 1.5 ns: 1.56x faster (-36%)

But please don't expect such large speedup on macro-benchmark.


tp_fastcall allows to remove the "cached args" optimization used in
various parts of Python core, old optimizations used in performance
critical code. This hack causes various kinds of complex bugs in
corner cases which can lead to crash in the worst case.


The patch to support tp_fastcall is tiny, but you should expect a long
list of tiny changes to replace tp_call with tp_fastcall in various
types.


Final bonus point: existing code (calling functions) doesn't need to
be modified (nor recompiled) to get speedup. Even if tp_call is
directly directly, fastcall will provide speedup, but only if it is
called only with positional arguments.

About the tp_call wrapper: keyword arguments require to convert a
Python dictionary to a C array which might be more expensive. I didn't
try to measure the performance, since this case is very rare. Almost
no C code calls functions with keyword arguments, just because it's
much more complex to pass keyword arguments, it requires too much C
code (and it's not simpler with fastcall, sorry).

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2017-01-13 Thread Python tracker 

ACTIVITY SUMMARY (2017-01-06 - 2017-01-13)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open5719 (+34)
  closed 35300 (+47)
  total  41019 (+81)

Open issues with patches: 2474 


Issues opened (53)
==

#10513: sqlite3.InterfaceError after commit
http://bugs.python.org/issue10513  reopened by benjamin.peterson

#26858: android: setting SO_REUSEPORT fails
http://bugs.python.org/issue26858  reopened by xdegaye

#28759: access to mkfifo, mknod and hard links is controled by SELinux
http://bugs.python.org/issue28759  reopened by xdegaye

#29183: Unintuitive error handling in wsgiref when a crash happens in 
http://bugs.python.org/issue29183  opened by jleclanche

#29184: skip tests of test_socketserver when bind() raises PermissionE
http://bugs.python.org/issue29184  opened by xdegaye

#29185: test_distutils fails on Android API level 24
http://bugs.python.org/issue29185  opened by xdegaye

#29187: Pickle failure is raising AttributeError and not PicklingError
http://bugs.python.org/issue29187  opened by Matt.Dodge

#29191: liblzma is missing from pcbuild.sln
http://bugs.python.org/issue29191  opened by Segev Finer

#29196: Remove old-deprecated plistlib features
http://bugs.python.org/issue29196  opened by serhiy.storchaka

#29197: Remove os.path.splitunc()
http://bugs.python.org/issue29197  opened by serhiy.storchaka

#29198: AsyncGenerator is missing from typing
http://bugs.python.org/issue29198  opened by Jelle Zijlstra

#29199: test_regrtest fails if PCBuild directory doesn't exist
http://bugs.python.org/issue29199  opened by ppperry

#29202: Improve dict iteration
http://bugs.python.org/issue29202  opened by rhettinger

#29204: Add code deprecations in ElementTree
http://bugs.python.org/issue29204  opened by serhiy.storchaka

#29205: col_offset for AsyncFunctionDef AST nodes is wrong
http://bugs.python.org/issue29205  opened by Jelle Zijlstra

#29207: make install fails on ARM
http://bugs.python.org/issue29207  opened by petevine

#29209: Remove old-deprecated ElementTree features
http://bugs.python.org/issue29209  opened by serhiy.storchaka

#29211: assertRaises with exceptions re-raised from a generator kills 
http://bugs.python.org/issue29211  opened by dalke

#29212: Python 3.6 logging thread name regression with concurrent.futu
http://bugs.python.org/issue29212  opened by desbma

#29213: python -m venv activate.bat has weird mix of line endings
http://bugs.python.org/issue29213  opened by Josh Wilson

#29216: Space saving step for the LRU cache
http://bugs.python.org/issue29216  opened by rhettinger

#29218: distutils: Remove unused install_misc class
http://bugs.python.org/issue29218  opened by ericvw

#29221: ABC Recursion Error on isinstance() with less than recursion l
http://bugs.python.org/issue29221  opened by Anthony Scopatz

#29225: distutils.command.install_lib.get_outputs() wrong with extensi
http://bugs.python.org/issue29225  opened by Elvis Stansvik

#29227: Reduce C stack consumption in function calls
http://bugs.python.org/issue29227  opened by haypo

#29230: Segfault in sqlite3
http://bugs.python.org/issue29230  opened by intchanter

#29232: Quiet Install
http://bugs.python.org/issue29232  opened by earl.maier

#29234: Disable inlining of _PyStack_AsTuple() to reduce the stack con
http://bugs.python.org/issue29234  opened by haypo

#29235: Allow profile/cProfile to be used as context managers
http://bugs.python.org/issue29235  opened by Thane Brimhall

#29236: 'an ASCII string of one or more PEM-encoded certificates' need
http://bugs.python.org/issue29236  opened by uzytkownik

#29237: Create enum for pstats sorting options
http://bugs.python.org/issue29237  opened by Thane Brimhall

#29238: Add more kwargs to cProfile's print_stats
http://bugs.python.org/issue29238  opened by Thane Brimhall

#29240: Implementation of the PEP 540: Add a new UTF-8 mode
http://bugs.python.org/issue29240  opened by haypo

#29241: sys._enablelegacywindowsfsencoding() don't apply to os.fsencod
http://bugs.python.org/issue29241  opened by JGoutin

#29242: Crash on GC when compiling PyPy
http://bugs.python.org/issue29242  opened by gumblex

#29243: --enable-optimizations makes common build commands always need
http://bugs.python.org/issue29243  opened by xiang.zhang

#29246: typing.Union raises RecursionError when comparing Union to oth
http://bugs.python.org/issue29246  opened by spiside

#29247: Document return value of epoll.poll
http://bugs.python.org/issue29247  opened by njs

#29248: os.readlink fails on Windows
http://bugs.python.org/issue29248  opened by eryksun

#29249: Pathlib glob ** bug
http://bugs.python.org/issue29249  opened by Jon Walsh

#29250: islink and stat follow_symlinks are inconsistent on Windows
http://bugs.python.org/issue29250  opened by eryksun

#29251: Class __dict__ is only a mapping proxy
http://bugs.python.org/issue29251  op

Re: [Python-Dev] Making the new dtrace support work on OS X

2017-01-13 Thread Lukasz Langa 
Looks like function-entry and function-return give you the C-level frame names 
for some reason. This was implemented on OS X 10.11 if that makes any 
difference. I will look at this in the evening, the laptop I'm on now is macOS 
Sierra with SIP which cripples dtrace.

> On Jan 12, 2017, at 5:08 AM, David Cournapeau  wrote:
> 
> Hi,
> 
> I was excited to see official dtrace support for python 3.6.0 on OS X, but I 
> have not been able to make it work:
> 
> 1. I built my own python from sources on OS X 10.9,  with the --with-dtrace 
> support
> 2. if I launch `python3.6 -q &` and then `sudo dtrace -l -P python$!`, I get 
> the following output:
> 
>ID   PROVIDERMODULE  FUNCTION NAME
>  2774 python48084 python3.6  _PyEval_EvalFrameDefault 
> function-entry
>  2775 python48084 python3.6  _PyEval_EvalFrameDefault 
> function-return
>  2776 python48084 python3.6   collect gc-done
>  2777 python48084 python3.6   collect gc-start
>  2778 python48084 python3.6  _PyEval_EvalFrameDefault line
> 
> Which looks similar but not the same as the example given in the doc at 
> https://docs.python.org/dev/howto/instrumentation.html#enabling-the-static-markers
>  
> 
> 
> 3. When I try to test anything with the given call_stack.d example, I can't 
> make it work at all:
> 
> """
> # script.py
> def start():
> foo()
> 
> def foo():
> pass
> 
> start()
> """
> 
> I am not very familiar with dtrace, so maybe I a missing a step, there is a 
> documentation bug, or it depends on which OS X version you are using ?
> 
> Thanks,
> David
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making the new dtrace support work on OS X

2017-01-13 Thread David Cournapeau 
On Fri, Jan 13, 2017 at 9:12 PM, Lukasz Langa  wrote:

> Looks like function-entry and function-return give you the C-level frame
> names for some reason. This was implemented on OS X 10.11 if that makes any
> difference. I will look at this in the evening, the laptop I'm on now is
> macOS Sierra with SIP which cripples dtrace.
>

On that hint, I tried on OSX 11.1. sw_vers says

ProductName: Mac OS X
ProductVersion: 10.11.6
BuildVersion: 15G1108

And there, the example worked as advertised w/ my build of 3.6.0. I will
try on more versions of OS X in our test lab.

David

>
> On Jan 12, 2017, at 5:08 AM, David Cournapeau  wrote:
>
> Hi,
>
> I was excited to see official dtrace support for python 3.6.0 on OS X, but
> I have not been able to make it work:
>
> 1. I built my own python from sources on OS X 10.9,  with the
> --with-dtrace support
> 2. if I launch `python3.6 -q &` and then `sudo dtrace -l -P python$!`, I
> get the following output:
>
>ID   PROVIDERMODULE  FUNCTION NAME
>  2774 python48084 python3.6  _PyEval_EvalFrameDefault
> function-entry
>  2775 python48084 python3.6  _PyEval_EvalFrameDefault
> function-return
>  2776 python48084 python3.6   collect
> gc-done
>  2777 python48084 python3.6   collect
> gc-start
>  2778 python48084 python3.6  _PyEval_EvalFrameDefault line
>
> Which looks similar but not the same as the example given in the doc at
> https://docs.python.org/dev/howto/instrumentation.
> html#enabling-the-static-markers
>
> 3. When I try to test anything with the given call_stack.d example, I
> can't make it work at all:
>
> """
> # script.py
> def start():
> foo()
>
> def foo():
> pass
>
> start()
> """
>
> I am not very familiar with dtrace, so maybe I a missing a step, there is
> a documentation bug, or it depends on which OS X version you are using ?
>
> Thanks,
> David
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> lukasz%40langa.pl
>
>
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com