[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: Oh :-( I didn't expect that I would have to declare PyTraceMalloc_NewReference() in object.h even if Py_LIMITED_API is defined... Python currently leaks too much things even if Py_LIMITED_API is defined. It's time to break the C API! :-) https://pythoncapi.r

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: New changeset c89a93271447ec65e83a1dc7605e62dbf272cafd by Victor Stinner in branch 'master': bpo-35053: Define _PyTraceMalloc_NewReference in object.h (GH-10107) https://github.com/python/cpython/commit/c89a93271447ec65e83a1dc7605e62dbf272cafd -- _

[issue35053] Enhance tracemalloc to trace properly free lists

Change by STINNER Victor : -- pull_requests: +9439 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: This change modifies _Py_NewReference() which is a very important function and it impacts Python performance. I prefer to keep the bug in Python 3.6 and 3.7 to not risk to introduce a regression. The bug exists since Python 3.4 and I'm the first one to spot

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: New changeset 6279c1c5003cd94b5e04e568ce3df7c4e8f1eaa3 by Victor Stinner in branch 'master': bpo-35053: Add Include/tracemalloc.h (GH-10091) https://github.com/python/cpython/commit/6279c1c5003cd94b5e04e568ce3df7c4e8f1eaa3 -- __

[issue35053] Enhance tracemalloc to trace properly free lists

Change by STINNER Victor : -- pull_requests: +9422 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: New changeset 9e00e80e213ebc37eff89ce72102c1f928ebc133 by Victor Stinner in branch 'master': bpo-35053: Enhance tracemalloc to trace free lists (GH-10063) https://github.com/python/cpython/commit/9e00e80e213ebc37eff89ce72102c1f928ebc133 --

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: While writing PR 10063, I was unhappy with _Py_NewReference() macro, and so I wrote bpo-35059 to convert it to a static inline function. -- ___ Python tracker _

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: Even if Python 3.6 and 3.7 are impacted by the bug, I propose to only fix Python 3.8 since the change modifies a _Py_NewReference() function which is critical for performance. -- ___ Python tracker

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: A little bit of history. I opened a bug 2 years ago but I closed it (lack of interest): https://github.com/vstinner/pytracemalloc/issues/2 I rewrote tracemalloc between version 0.9 and 1.0. In tracemalloc 0.9, there was an API to track free lists. Here is th

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: Python 3.8 uses many free lists: https://pythondev.readthedocs.io/cpython_impl_optim.html#free-lists Attached dict_wrong_traceback.py shows the bug on the dictionary of an object: $ ./python ~/dict_wrong_traceback.py File "/home/vstinner/dict_wrong_traceba

[issue35053] Enhance tracemalloc to trace properly free lists

STINNER Victor added the comment: > Is performance overhead negligible? Thank you for asking the most important question :-) I ran this microbenchmark: make distclean ./configure --enable-lto make ./python -m venv env env/bin/python -m pip install perf sudo env/bin/python -m perf system tune

[issue35053] Enhance tracemalloc to trace properly free lists

INADA Naoki added the comment: Is performance overhead negligible? -- nosy: +inada.naoki ___ Python tracker ___ ___ Python-bugs-lis

[issue35053] Enhance tracemalloc to trace properly free lists

Change by Serhiy Storchaka : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https

[issue35053] Enhance tracemalloc to trace properly free lists

Change by STINNER Victor : -- keywords: +patch pull_requests: +9399 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-l

[issue35053] Enhance tracemalloc to trace properly free lists

New submission from STINNER Victor : CPython uses many "free lists": list of "deallocated" objects which are kept alive to optimize allocation of new objects. For example, the builtin list type has a free list. Problem: tracemalloc only traces the memory allocation when the object is created