New submission from STINNER Victor: Attached patch rewrites PyUnicode_FromFormatV(): * simplify the code: replace 4 steps with one unique step. PyUnicode_Format() has the same design. It avoids to store intermediate results which require to allocate an array of pointers in the heap. * use the _PyUnicodeWriter API for speed (and its convinient API): overallocate the buffer to reduce the number of "realloc()" * Implement "width" and "precision" in Python, don't rely on sprintf(). It avoids to need of a temporary buffer allocated on the heap: only use a small buffer allocated in the stack. * Detect integer overflow when parsing width and precision, as done in PyUnicode_Format() * Add _PyUnicodeWriter_WriteCstr() function * Split PyUnicode_FromFormatV() into smaller functions: add unicode_fromformat_arg(). It requires to copy vargs using Py_VA_COPY: without Py_VA_COPY, the function does crash. I don't understand why. * Inline parse_format_flags(): the format of an argument is now only parsed once, it's no more needed to have a subfunction. * Optimize PyUnicode_FromFormatV() for characters between two arguments: search the next "%" and copy the substring in one chunk, instead of copying character per character. * Replace "prec too big" with "precision too big" in error messages
_tescapi.test_string_from_format() is 20% faster with the patch according to timeit. I don't know how to write better benchmarks because PyUnicode_FromV() is not exposed in Python. I wrote a benchmark using ctypes to call the function, but it looks like the ctypes overhead is too high. I wrote the patch to simplify the code, but it may be faster thanks to the _PyUnicodeWriter API and some optimizations implemented in the patch. ---------- files: unicode_fromformat.patch keywords: patch messages: 172138 nosy: haypo priority: normal severity: normal status: open title: Rewrite PyUnicode_FromFormatV() to use the _PyUnicodeWriter API type: performance versions: Python 3.4 Added file: http://bugs.python.org/file27440/unicode_fromformat.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue16147> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
