Neil Schemenauer added the comment:
As I suspected, the performance impact is significant (although pretty small).
Based on Linux perf, it looks like the extra test+branch in _Py_Dealloc adds
about 1% overhead. pyperformance shows something similar, see attached reports
(pypref
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file50220/perf-annotate-trash.txt
___
Python tracker
<https://bugs.python.org/issue44897>
___
___
Pytho
Neil Schemenauer added the comment:
Based on some testing, I think an extra type slot is not worth the extra
complication. I made some small improvements to _Py_Dealloc and now the
performance seems a bit better. Basically, I expanded _PyObject_IS_GC() to put
the most common branches
Change by Neil Schemenauer :
--
nosy: +pablogsal, tim.peters, vstinner
___
Python tracker
<https://bugs.python.org/issue44897>
___
___
Python-bugs-list mailin
Neil Schemenauer added the comment:
Another small correction, _PyType_IS_GC() checks only the type flag
(Py_TPFLAGS_HAVE_GC). _PyObject_IS_GC() checks both the type flag and calls
tp_is_gc(), if it exists. tp_is_gc() is wart, IMHO, and it would be nice if we
can kill it off so only the
Neil Schemenauer added the comment:
[Larry]
> The one thing I guess I never mentioned is that building and working with the
> prototype was frightful; it had both Python code and C code, and it was
> fragile and hard to get working.
I took Larry's PR and did a fair amount of cl
New submission from Neil Schemenauer :
There is a typo in the radix tree obmalloc code, spotted by Inada Naoki.
-#define MAP_TOP_MASK (MAP_BOT_LENGTH - 1)
+#define MAP_TOP_MASK (MAP_TOP_LENGTH - 1)
This should be fixed both in the main branch and in 3.10.x.
--
assignee: methane
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27322
pull_request: https://github.com/python/cpython/pull/29051
___
Python tracker
<https://bugs.python.org/issue45
New submission from Neil Schemenauer :
Given this feedback:
https://github.com/python/cpython/pull/14474/files#r725488766
it is perhaps not so safe to assume that only the lower 48 bits of virtual
addresses are significant. I had the idea that Go made similar assumptions but
now I'
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27329
pull_request: https://github.com/python/cpython/pull/29062
___
Python tracker
<https://bugs.python.org/issue45
Neil Schemenauer added the comment:
I have not yet been able to reproduce methane's crash. My guess it it's not
related.
An explanation of what I think the impact of this bug is:
The radix tree is used to determine if memory is from obmalloc or from the
system malloc (i.e re
Neil Schemenauer added the comment:
New changeset 311910b31a4bd94dc79298388b7cb65ca5546438 by Neil Schemenauer in
branch 'main':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
https://github.com/python/cpython/commit/311910b31a4bd94dc79298388b7cb6
Neil Schemenauer added the comment:
New changeset 1cdac61065e72db60d26e03ef9286d2743d7000e by Miss Islington (bot)
in branch '3.10':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) (GH-29122)
https://github.com/python/cpython/commit/1cdac61065e72db60d26e03ef9286d
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Neil Schemenauer :
Some makefile rules don't work if you build in a separate folder.
--
messages: 404671
nosy: nascheme
priority: normal
severity: normal
stage: patch review
status: open
title: Run smelly.py and multissltests.py from $(srcdir)
type: beh
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27415
pull_request: https://github.com/python/cpython/pull/29138
___
Python tracker
<https://bugs.python.org/issue45
Neil Schemenauer added the comment:
Closing since I believe the bug is fixed and there is an appropriate unit test.
--
assignee: -> nascheme
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tra
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Neil Schemenauer :
Updating the frozen module "__hello__" code inside Python/frozen.c is currently
a manual process. That's a bit tedious since it adds some extra work in the
case that bytecode changes are made. I've created a small script and a
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +17558
pull_request: https://github.com/python/cpython/pull/18174
___
Python tracker
<https://bugs.python.org/issue39
Neil Schemenauer added the comment:
Eddie mentions in the PR about using memory arenas to contain immortal objects.
I think it could be worth investigating further. With the current PR, the
immortal status is dependent on the value of the refcnt field of the object.
Using immortal arenas
Neil Schemenauer added the comment:
A few thoughts
First, the PR is well done. The changes are pretty minimal and are well
localized. Thanks Eddie for writing clean code and responding to review
requests.
My feeling is that, in current state, this still should not get merged or at
least
Neil Schemenauer added the comment:
> immutability vs immutable object headers
Sorry, I meant to write "immortal vs immutable".
--
___
Python tracker
<https://bugs.pytho
Neil Schemenauer added the comment:
I resurrected an old thread on Discourse that seems quite relevant to this PR:
https://discuss.python.org/t/switching-from-refcounting-to-libgc/1641/35?u=nas
--
___
Python tracker
<https://bugs.python.
Neil Schemenauer added the comment:
Maybe something to do the configuration of the machine, so that TCP/UDP socket
networking if failing? Two things to try:
- give the "python" process a SIGINT signal and you should get a traceback
showing where it is hanging. Or you could use
Neil Schemenauer added the comment:
New changeset ffa55d21b4a86ad8b4a43a9f597151e526541130 by Neil Schemenauer in
branch 'master':
bpo-39448: Add regen-frozen makefile target. (GH-18174)
https://github.com/python/cpython/commit/ffa55d21b4a86ad8b4a43a9f597151
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Neil Schemenauer :
The FileSystem class is missing a skip() method. If the file system doesn't
support Unicode filenames, the test crashes.
File "/home/nas/src/cpython/Lib/test/test_importlib/fixtures.py", line 221,
in unicode_filename
self.skip(&q
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23391
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
nosy: +nascheme, nascheme
nosy_count: 9.0 -> 10.0
pull_requests: +23392, 23393
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 9.0 -> 10.0
pull_requests: +23392
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
New changeset 50288aa8c955f66ab67a7dadf250ea5f4238eb67 by Neil Schemenauer in
branch 'master':
bpo-43288: Fix bug in test_importlib test. (GH-24612)
https://github.com/python/cpython/commit/50288aa8c955f66ab67a7dadf250ea
Change by Neil Schemenauer :
--
pull_requests: +23395
pull_request: https://github.com/python/cpython/pull/24614
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
New changeset 84f7afe65c29330f3ff8e318e054b96554a2dede by Neil Schemenauer in
branch 'master':
Fix failed merge of bpo-43288. (GH-24614)
https://github.com/python/cpython/commit/84f7afe65c29330f3ff8e318e054b9
Change by Neil Schemenauer :
--
pull_requests: +23397
pull_request: https://github.com/python/cpython/pull/24616
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
New changeset 44fe32061d60f4bd9c4fa48c24e3e4ba26033dba by Neil Schemenauer in
branch '3.9':
[3.9] bpo-43288: Fix bug in test_importlib test. (GH-24616)
https://github.com/python/cpython/commit/44fe32061d60f4bd9c4fa48c24e3e4
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file49833/perf_compare_noradix.txt
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Pytho
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file49834/perf_compare_radix4x.txt
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Pytho
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 3.0 -> 4.0
pull_requests: +23486
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
I believe the line table format got changed but the frozen code didn't get
re-generated. If you try to call co_lines() on the __hello__ code, Python
crashes.
>>> import __hello__
Hello world!
>>> co = __hello__.__spec__.loa
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 7.0 -> 8.0
pull_requests: +23485
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23484
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
New submission from Neil Schemenauer :
In bug #43372, we didn't notice that the code for the __hello__ module was not
re-generated. Things seems to be okay but the line number table was corrupted.
It seems a good idea to add a small test to ensure that doesn't happen again.
I
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23490
pull_request: https://github.com/python/cpython/pull/24712
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
pull_requests: +23491
pull_request: https://github.com/python/cpython/pull/24714
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
pull_requests: +23492
pull_request: https://github.com/python/cpython/pull/24714
___
Python tracker
<https://bugs.python.org/issue42
Neil Schemenauer added the comment:
I think it may be related to bpo-41561. There is a bug in the Ubuntu tracker
as well:
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878
I agree with the temporary fix to use "ubuntu-18.04" for CI testing.
--
nosy:
New submission from Neil Schemenauer :
While I was fixing the regen-frozen issue, I noticed it seems unnecessary to
have regen-stdlib-module-names separate from regen-all. Maybe Victor knows why
it needs to be separate. If it doesn't need to be separate, the CI scripts can
be sli
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23494
pull_request: https://github.com/python/cpython/pull/24713
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
It seems it is enough to make a new commit. The CI seems to re-base and re-run
the PR. At least, it worked on two of my PRs.
--
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
pull_requests: +23531
pull_request: https://github.com/python/cpython/pull/24759
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
pull_requests: +23532
pull_request: https://github.com/python/cpython/pull/24759
___
Python tracker
<https://bugs.python.org/issue42
Change by Neil Schemenauer :
--
assignee: -> nascheme
___
Python tracker
<https://bugs.python.org/issue43372>
___
___
Python-bugs-list mailing list
Unsubscrib
Neil Schemenauer added the comment:
New changeset 87ec26b812e9c4095c017dc60f246eda37b83ab2 by Neil Schemenauer in
branch 'master':
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
https://github.com/python/cpython/commit/87ec26b812e9c4095c017dc60f246e
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset 87ec26b812e9c4095c017dc60f246eda37b83ab2 by Neil Schemenauer in
branch 'master':
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
https://github.com/python/cpython/commit/87ec26b812e9c4095c017dc60f246e
Neil Schemenauer added the comment:
Not sure the proper place to discuss this but I wonder if putting this stdlib
module names list in the executable is the best idea. The list of available
stdlib modules could change after compiling Python. I understand you don't
want to dynami
New submission from Neil Schemenauer :
It is great that access to ob_type has been cleaned up to use an access macro.
There are two spots that still need fixing.
I think we should do something to help avoid this kind of thing slipping into
the code in the future. E.g. a special build flag
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +21784
pull_request: https://github.com/python/cpython/pull/22829
___
Python tracker
<https://bugs.python.org/issue42
Neil Schemenauer added the comment:
> I don't understand the rationale for this change in depth, but
> does the benefit outweigh (yet another) backwards incompatibility?
I think you can have it both ways. Do you want a C API that is
stable over a long period of CPython release
Neil Schemenauer added the comment:
Correction: I think you *cannot* have it both ways.
--
___
Python tracker
<https://bugs.python.org/issue39573>
___
___
Pytho
Neil Schemenauer added the comment:
I believe this is not needed anymore. Closing.
--
resolution: -> rejected
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
This has been resolved using Py_TYPE() and Py_SET_TYPE().
--
resolution: -> out of date
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Neil Schemenauer :
--
resolution: -> out of date
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
I think the comments are correct in that it is used to create a new statically
linked interpreter that includes a user provided extension module. We could
include python.o inside the libpythonXX.a file but then I think it breaks if
you are embedding (e.g
Neil Schemenauer added the comment:
> sys.getsizeof(3.14) is 24. And it becomes 32 byte in 16byte aligned
> pymalloc. (+33%)
I've been doing some reading and trying to understand this issue. My
understanding is that malloc() needs to return pointers that are 16-byte
aligned o
Neil Schemenauer added the comment:
We now have a concrete use case. ;-)
My idea was that we can introduce a new, CPython internal API that
aligns on 8-byte boundaries (or takes alignment as a parameter). The
API would be a stop-gap measure. We can use the API to reduce
the overhead for
New submission from Neil Schemenauer :
In the process of working on some garbage collector/obmalloc experiments, I
noticed what seems to be a quirk about PyType_GenericAlloc(). It calls:
size = _PyObject_VAR_SIZE(type, nitems+1);
Note the "+1" which is documented as "f
Neil Schemenauer added the comment:
Updated patch is attached. It appears that the extra item is only needed if
Py_TPFLAGS_TYPE_SUBCLASS set. In all other cases, it seems we don't need the
extra space for the sentinel. At least, the unit tests pass with this change.
It could be
Neil Schemenauer added the comment:
I suggest we change PyCode_New() back in the next beta, despite that change
breaking Cython again. We could work with them so that they have a new release
with a "PY_VERSION_HEX > 0x030800b1" test in it. Try to get that Cython version
re
New submission from Neil Schemenauer :
This patch implements an alternative version of obmalloc's address_in_range().
It uses a radix tree to map the areas of memory covered by obmalloc arenas.
pymalloc_free() uses address_in_range() to determine if a block of memory is
controll
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file48445/pyperf_radix_compare.txt
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Pytho
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file48446/obmalloc_overhead.py
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Python-bug
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +14291
pull_request: https://github.com/python/cpython/pull/14474
___
Python tracker
<https://bugs.python.org/issue37
New submission from Neil Schemenauer :
This is a small but measurable optimization for _PyObject_Malloc and
_PyObject_Free. It slows down _Py_GetAllocatedBlocks a bit but I think that's
a price worth paying.
--
components: Interpreter Core
messages: 347602
nosy: nascheme
pri
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +14486
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/14680
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
PyPerformance result, using --enable-optimizations
+-+-+--+
| Benchmark | base| no-allocated
Neil Schemenauer added the comment:
New changeset 5d25f2b70351fc6a56ce5513ccf5f58556c18837 by Neil Schemenauer in
branch 'master':
bpo-37537: Compute allocated blocks in _Py_GetAllocatedBlocks() (#14680)
https://github.com/python/cpython/commit/5d25f2b70351fc6a56ce5513ccf5f5
Neil Schemenauer added the comment:
> Decreasing the total wall time for a default --enable-optimizations build
> would
> be a good thing for everyone, provided the resulting interpreter remains
> "effectively similar" in speed. If you somehow manage to find som
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +14501
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/14702
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
I tweaked the list of unit tests a little more, trying to incorporate some from
your Docker build settings. Not sure what's going on with the pickle results.
Below are new pyperformance runs, comparing my PR to the "master" version it i
Neil Schemenauer added the comment:
+-+---+-+
| Benchmark | task-all2 | task-short7 |
+=+===+=+
| 2to3| 304 ms| 305 ms
Neil Schemenauer added the comment:
Thanks for the feedback. I agree that putting the tests in regrtest is better.
I've made the following changes:
- the --pgo option now uses the shorter list of tests by default
- I added --pgo-extended as a regrtest option that enables the old beh
Neil Schemenauer added the comment:
I changed configure.in to use AC_ARG_VAR instead. So, you can override it as
you suggest:
./configure [..] PROFILE_TASK="-m test --pgo-extended"
--
___
Python tracker
<https://bugs.python.o
Neil Schemenauer added the comment:
New changeset 4e16a4a3112161a5c6981c0588142d4a4673a934 by Neil Schemenauer in
branch 'master':
bpo-36044: Reduce number of unit tests run for PGO build (GH-14702)
https://github.com/python/cpython/commit/4e16a4a3112161a5c6981c0588142d
New submission from Neil Schemenauer :
Add a new support decorator, @skip_if_pgo_task and then use it to mark test
cases. I suspect the PGO task works well if it can exercise common code paths.
Running long tests likely have rapidly diminishing benefits. The instrumented
PGO executable
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +14772
pull_request: https://github.com/python/cpython/pull/15009
___
Python tracker
<https://bugs.python.org/issue37
Neil Schemenauer added the comment:
I think expanding the list of tests used by --pgo is fine, as long as we put a
little thought into each one we add. The ones added by Steve look fine to me.
It seems useful to run a profiler and see if there are any unusually expensive
tests being added
Neil Schemenauer added the comment:
New changeset 52a48e62c6a94577152f9301bbe5f3bc806cfcf1 by Neil Schemenauer in
branch 'master':
bpo-37707: Exclude expensive unit tests from PGO task (GH-15009)
https://github.com/python/cpython/commit/52a48e62c6a94577152f9301bbe5f3
Neil Schemenauer added the comment:
Closing as I think PR 14702 mostly resolves this. The gc_collect() issue has
not been resolved but based on some investigation, I think it is not worth the
effort to try to reduce the time spend in that function. Reducing the number
of unit tests being
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset 382cb85401bb010ead411c0532499ffe16c3cf27 by Neil Schemenauer
(Miss Islington (bot)) in branch '3.8':
bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024)
https://github.com/python/cpyt
New submission from Neil Schemenauer :
I find it annoying and surprising that "make clean" does not remove the PGO
data. If you change a source file, running "make clean" and "make" should
result in a newly built executable, IMHO. As it is now, you usually
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +14790
pull_request: https://github.com/python/cpython/pull/15033
___
Python tracker
<https://bugs.python.org/issue37
New submission from Neil Schemenauer:
After a lot of head scratching, I have discovered why tests are failing for me.
E.g. "./python Lib/test/test_datetime.py" results in
AttributeError: module 'bisect' has no attribute 'bisect_right'
Running tests this way cau
Neil Schemenauer added the comment:
New changeset e38d12ed34870c140016bef1e0ff10c8c3d3f213 by Neil Schemenauer in
branch 'master':
bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit.
(#1908)
https://github.com/python/cpyt
Changes by Neil Schemenauer :
--
pull_requests: +3352
___
Python tracker
<http://bugs.python.org/issue17852>
___
___
Python-bugs-list mailing list
Unsubscribe:
Neil Schemenauer added the comment:
The key is how the test is run. If you run:
$ ./python Lib/test/.py
Then the Lib/test gets added to the first part of sys.path. Then "import
bisect" imports the Lib/test/bisect.py module rather than Lib/bisect.py.
Obvious options seem to be:
Neil Schemenauer added the comment:
New changeset db564238db440d4a2d8eb9d60ffb94ef291f6d30 by Neil Schemenauer in
branch 'master':
Revert "bpo-17852: Maintain a list of BufferedWriter objects. Flush them on
exit. (#1908)" (#3337)
https://github.com/p
101 - 200 of 375 matches
Mail list logo
