New submission from STINNER Victor:
Python emits ResourceWarning when an object using limited resource is destroyed
without being explicitly closed: files, sockets, etc. The problem is that it's
hard to find where the object comes from, since the warning can occur very
late, in the garbage collector, etc.
I propose to reuse tracemalloc.get_object_traceback() to show were the object
was allocated, when tracemalloc traces memory allocations. In practice, I
propose to add a new "source" parameter to warnings.showwarning().
Attached patch:
* Add a new keyword-only source parameter to warnings.showwarning()
* Add C function PyErr_ResourceWarning() to pass source
* showwarning() uses tracemalloc.get_object_traceback() to get the traceback
were the object was allocated
* Modify socket.socket, io.FileIO and os.scandir destructor to use
PyErr_ResourceWarning()
Backward-compatibility problem: The C function PyErr_ResourceWarning() always
call warnings.showwarning() with the keyword parameter source. If an
application replaces the warnings.showwarning() function, it will probably fail
because it doesn't know the source parameter.
I don't know how to handle this backward compatibility issue.
The patch is incomplete, it's not possible yet to emit a warning in pure Python
with a source parameter.
x.py script used for examples below:
-----------------
import warnings
import os
import socket
def func2():
#f=open("/etc/issue")
#f=os.scandir('.')
f=socket.socket()
f=None
def func():
func2()
func()
-----------------
Output with Python 3.5:
-----
x.py:9: ResourceWarning: unclosed <socket.socket fd=3, ...>
f=None
-----
Output with -X tracemalloc=5 command line option and patched Python 3.6:
-----
x.py:9: ResourceWarning: unclosed <socket.socket fd=3,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
laddr=('0.0.0.0', 0)>
f=None
Object allocated at (most recent call first):
File "x.py", lineno 8
f=socket.socket()
File "x.py", lineno 12
func2()
File "x.py", lineno 14
func()
-----
It's much easier to understand where the warning comes from, no? At x.py:8,
line "f=socket.socket()".
Note: the traceback doesn't contain the function name, since tracemalloc only
stores filename and line number.
See also the issue #26564 "Malloc debug hooks: display memory block traceback
on error".
For Python < 3.6, I wrote "res_warn.py" script which monkey-patches io.FileIO
and socket.socket to implement something similar. The script is a fragile hack.
I would prefer to have the feature built-in Python.
https://bitbucket.org/haypo/misc/src/0a40f27360424145bad0f9b62c9e9148ffdbb169/python/res_warn.py
----------
files: warnings_tracemalloc.patch
keywords: patch
messages: 261812
nosy: haypo, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Use tracemalloc to display the traceback where an object was allocated
when a ResourceWarning is emitted
versions: Python 3.6
Added file: http://bugs.python.org/file42171/warnings_tracemalloc.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26567>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
