[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

STINNER Victor added the comment: I'm no more interested to work on this issue, and it's not clear that _PyUnicodeWriter is always faster. Switch from a list to _PyUnicodeWriter on a specific event would make the code much more complex. I prefer to just close the issue. -- resolution:

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Serhiy Storchaka added the comment: See benchmark results in issue15381 (the patch is not applicable to StringIO). These numbers show that resize strategy can be much slower append/join strategy on Windows. -- nosy: +storchaka ___ Python tracker

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Antoine Pitrou added the comment: > _PyUnicodeWriter is almost always faster Actually, PyAccu is consistently faster for the "writer" case, while _PyUnicodeWriter is faster for the "writer-reader" case. This is not because of PyAccu, but because of the way StringIO uses it: when e.g. readline(

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

STINNER Victor added the comment: "PyAccu looks much more appropriate than _PyUnicodeWriter, because it is always faster, except to write 100.000 very long lines." Oh... I added colors to my tool, but there was a bug: I used the wrong colors... It's just the opposite. _PyUnicodeWriter is almo

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

STINNER Victor added the comment: > Victor, your benchmark is buggy (it writes one character at a time). Oh, it's not what I wanted to test. I attach a new benchmark. Here are the results. PyAccu looks much more appropriate than _PyUnicodeWriter, because it is always faster, except to write 1

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Antoine Pitrou added the comment: Victor, your benchmark is buggy (it writes one character at a time). You should apply the following patch: $ diff -u bench_stringio_orig.py bench_stringio.py --- bench_stringio_orig.py 2012-08-11 12:02:16.528321958 +0200 +++ bench_stringio.py 2012-08-11

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Antoine Pitrou added the comment: > > I would like to know why that is the case. > > If PyUnicode_Join is not optimal, then perhaps we should > > better optimize it. > > I don't know. _PyUnicodeWriter overallocates its buffer (+25%). It may > reduce the number of realloc(), and so the number of

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

STINNER Victor added the comment: > I would like to know why that is the case. > If PyUnicode_Join is not optimal, then perhaps we should > better optimize it. I don't know. _PyUnicodeWriter overallocates its buffer (+25%). It may reduce the number of realloc(), and so the number of times that

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Antoine Pitrou added the comment: > Also, you should post benchmarks with tiny strings as well. Oops, sorry, they are already there. Thanks for the numbers. -- ___ Python tracker __

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Antoine Pitrou added the comment: > It provides better performance when writing non-ASCII strings. I would like to know why that is the case. If PyUnicode_Join is not optimal, then perhaps we should better optimize it. Also, you should post benchmarks with tiny strings as well. -- __

[issue15612] Rewrite StringIO to use the _PyUnicodeWriter API

Changes by STINNER Victor : -- title: Rewriter StringIO to use the _PyUnicodeWriter API -> Rewrite StringIO to use the _PyUnicodeWriter API ___ Python tracker ___ __