[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: New changeset efde51ad54c58353f25ff80c8d30dbee82ef33a3 by INADA Naoki in branch '3.6': bpo-29949: Fix set memory usage regression (GH-945) https://github.com/python/cpython/commit/efde51ad54c58353f25ff80c8d30dbee82ef33a3 -- _

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by INADA Naoki : -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker ___ ___ Python-bugs-li

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by INADA Naoki : -- pull_requests: +1127 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: New changeset e82cf8675bacd7a03de508ed11865fc2701dcef5 by INADA Naoki in branch 'master': bpo-29949: Fix set memory usage regression (GH-943) https://github.com/python/cpython/commit/e82cf8675bacd7a03de508ed11865fc2701dcef5 -- __

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by INADA Naoki : -- pull_requests: +1125 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by Cameron Simpson : -- nosy: +cameron ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue29949] sizeof set after set_merge() is doubled from 3.5

Raymond Hettinger added the comment: Do you want to prepare a PR for me? I not yet set-up with the ways of Github. Please limit the PR to just unwinding the refactoring in the simplest possible way.. If in the future you want to chat about speed/space trade-offs, we can do that offline. I'

[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: I agree. Before thinking about rebalance between size and speed, resolving regression is important. -- ___ Python tracker ___

[issue29949] sizeof set after set_merge() is doubled from 3.5

Raymond Hettinger added the comment: I think the best thing to do is to undo the refactoring in https://github.com/python/cpython/commit/4897300276d870f99459c82b937f0ac22450f0b6 . It is was intended be neutral but did affect set_update_internal() for small sets. --

[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: > frozenset(range(n)) is slightly larger in 3.7 than in 3.6. It is 4 times > larger for about 10% of sizes. This is intensional: https://github.com/python/cpython/commit/5cd87a8d61246b0a6233bfb8503d4718b693cef0 load factor is reduced from 66% to 60%. (-10%)

[issue29949] sizeof set after set_merge() is doubled from 3.5

Serhiy Storchaka added the comment: $ ./python -c 'N = 6000; from sys import getsizeof; s = [getsizeof(frozenset(range(n))) for n in range(N)]; print( [(n, s[n]) for n in range(N) if not n or s[n] != s[n-1]] )' 3.5: [(0, 112), (6, 368), (22, 1136), (86, 4208), (342, 16496), (1366, 65648), (5

[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: See set_update_internal(). https://github.com/python/cpython/blob/master/Objects/setobject.c#L969-L1016 This happens only when iterable is set or dict. >>> import sys >>> sys.getsizeof(set(range(10))) 736 >>> sys.getsizeof(set(set(range(10 1248 >>> sys.getsize

[issue29949] sizeof set after set_merge() is doubled from 3.5

Raymond Hettinger added the comment: Hmm, I wonder why I'm not seeing the same sizes you are seeing. $ cat setsize.py from sys import getsizeof print( [getsizeof(frozenset(range(n))) for n in range(20)] ) $ python3.4 setsize.py [224, 224, 224, 224, 224, 224, 736, 736, 736, 736, 736, 736, 736, 73

[issue29949] sizeof set after set_merge() is doubled from 3.5

Raymond Hettinger added the comment: I'll look at this over the next week or two. I don't really like the proposed patch at all but will carefully think through the speed/space trade-offs. -- ___ Python tracker _

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by Raymond Hettinger : -- assignee: -> rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue29949] sizeof set after set_merge() is doubled from 3.5

INADA Naoki added the comment: https://gist.github.com/methane/8faf12621cdb2166019bbcee65987e99 This patch fixes the regression. But I think it is still larger than idiomatic. See this code: code: minused *= 2; /* Find the smallest table size > minused. */ /* XXX speed-up with intr

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by Jan Gosmann : -- nosy: +jgosmann ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue29949] sizeof set after set_merge() is doubled from 3.5

Changes by Serhiy Storchaka : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:

[issue29949] sizeof set after set_merge() is doubled from 3.5

New submission from INADA Naoki: (original thread is https://mail.python.org/pipermail/python-list/2017-March/720391.html) https://github.com/python/cpython/commit/4897300276d870f99459c82b937f0ac22450f0b6 this commit doubles sizeof set object created by set_merge(). It is used by constructor o