[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-18 Thread STINNER Victor
Changes by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-18 Thread Roundup Robot
Roundup Robot added the comment: New changeset 093b9838a41c by Victor Stinner in branch 'default': Issue #19581: Change the overallocation factor of _PyUnicodeWriter on Windows http://hg.python.org/cpython/rev/093b9838a41c -- nosy: +python-dev ___ Pyt

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-18 Thread STINNER Victor
STINNER Victor added the comment: > Your patch implies that the two only supported OSes are Linux and Windows :-) It more means that Windows memory allocator is different to the one used on all other operating systems. Well, if you are not convinced, we can keep the overallocation factor of 25

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread STINNER Victor
STINNER Victor added the comment: > Did you find a difference for small strings vs. large strings? When I replaced PyAccu with PyUnicodeWriter for str%args and str.format(args), I ran a lot of benchmarks with short, medium and large strings. See for example: http://bugs.python.org/file25687/REP

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread Antoine Pitrou
Antoine Pitrou added the comment: Did you find a difference for small strings vs. large strings? -- nosy: +tim.peters ___ Python tracker ___ _

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread STINNER Victor
STINNER Victor added the comment: I chose 25% on Linux after some micro-benchmarks on str%args and str.format(args). If the buffer is too large, the final resize (because PyUnicodeObject must have the exact size) is slow. I suppose that realloc() can avoid copying data if the new is is very cl

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: Many collections in other programming languages use overallocating rate 100%. Perhaps overallocating rate 12.5% for Python lists is good compromise between speed and memory usage (as far as Python lists have no explicit resize method), but for short-living o

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread Antoine Pitrou
Antoine Pitrou added the comment: Your patch implies that the two only supported OSes are Linux and Windows :-) To be honest I think we should have one single overallocation factor for all OSes. If 50% is ok on Linux too, then let it be 50%. (did you run stringbench to see if it made a differe

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

2013-11-14 Thread STINNER Victor
New submission from STINNER Victor: PyUnicodeWriter currently overallocates the internal buffer by 25%. On Windows, PyUnicodeWriter is slower than PyAccu API. With an overallocation factor of 50%, PyUnicodeWriter is fastter. See this message for the benchmark: http://bugs.python.org/issue19513