[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Fixed in 2.7 and 3.2; extra tests ported to 3.3 and default. Reclosing. -- resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue14700] Integer overflow in classic string formatting

Roundup Robot added the comment: New changeset 102df748572d by Mark Dickinson in branch '3.2': Issue #14700: Fix buggy overflow checks for large precision and width in new-style and old-style formatting. http://hg.python.org/cpython/rev/102df748572d New changeset 79ea0c84152a by Mark Dickinson

[issue14700] Integer overflow in classic string formatting

Roundup Robot added the comment: New changeset 21fb1767e185 by Mark Dickinson in branch '2.7': Issue #14700: Fix buggy overflow checks for large precision and width in new-style and old-style formatting. http://hg.python.org/cpython/rev/21fb1767e185 -- _

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: Mark, can I help? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue14700] Integer overflow in classic string formatting

STINNER Victor added the comment: For your information, I fixed recently PyUnicode_FromFormatV() to detect overflows on width and precision: changeset: 79543:d1369daeb9ec user:Victor Stinner date:Sat Oct 06 23:05:00 2012 +0200 files: Objects/unicodeobject.c description:

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Thanks for reviewing. I was being lazy with the checks; I'll fix that. Agreed that it's worth forward porting the tests to 3.3 and 3.4; I'll do that. -- ___ Python tracker _

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: Only one comment. test_formatting_huge_precision should use not sys.maxsize, but _testcapi.INT_MAX. Other tests can use _testcapi.PY_SSIZE_T_MAX. I think this tests are worth to add for 3.3 and 3.4. Your old test for this bug (064c2d0483f8) actually does not

[issue14700] Integer overflow in classic string formatting

Changes by Mark Dickinson : Removed file: http://bugs.python.org/file27472/formatting-overflow-3.2.patch ___ Python tracker ___ ___ Python-bug

[issue14700] Integer overflow in classic string formatting

Changes by Mark Dickinson : Added file: http://bugs.python.org/file27473/formatting-overflow-3.2.patch ___ Python tracker ___ ___ Python-bugs-

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: And for 3.2 -- Added file: http://bugs.python.org/file27472/formatting-overflow-3.2.patch ___ Python tracker ___ ___

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Here's a patch for 2.7. -- Added file: http://bugs.python.org/file27471/formatting-overflow-2.7.patch ___ Python tracker ___ ___

[issue14700] Integer overflow in classic string formatting

Changes by Mark Dickinson : -- resolution: fixed -> status: closed -> open versions: -Python 3.3 ___ Python tracker ___ ___ Python-b

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Re-opening: this should probably also be fixed in 2.7 and 3.2. See issue 16096 for discussion. -- ___ Python tracker ___ _

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > I somehow doubt that this micro-optimization is going to have any noticeable > effect in real code. Agree. I just found this bug, trying to optimize the code. -- ___ Python tracker

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Sure, I realize that, but I prefer not to be sloppy in the overflow check, and to use the same formula that's already used in stringlib. I somehow doubt that this micro-optimization is going to have any noticeable effect in real code. -- ___

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: Mark, I deliberately have not used the exact formula for the overflow. Comparison with the constant is much cheaper than division or multiplication. Microbencmark: ./python -m timeit -s 'f="%.1234567890s"*100;x=("",)*100' 'f%x' Before changeset 064c2d0483

[issue14700] Integer overflow in classic string formatting

Changes by Mark Dickinson : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue14700] Integer overflow in classic string formatting

Roundup Robot added the comment: New changeset 064c2d0483f8 by Mark Dickinson in branch 'default': Issue #14700: Fix two broken and undefined-behaviour-inducing overflow checks in old-style string formatting. Thanks Serhiy Storchaka for report and original patch. http://hg.python.org/cpython/

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > I'll add tests and apply this later. Well, look at test_crasher in Lib/test/test_struct.py. -- ___ Python tracker ___

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Still, I think it would be useful to have some tests that exercise the overflow branches. (If those tests had existed before, then this issue would probably already have been found and fixed, since clang could have detected the undefined behaviour resulting

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > 32-bit Linux (Ubuntu), gcc 4.6. Sorry, gcc 4.4. -- ___ Python tracker ___ ___ Python-bugs-list

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > Any chance of some tests? :-) Even a test for struct tests only struct.calcsize on this specific case. For string formatting has no such function, on most platforms testing would be a memory overflow. -- ___ Pyt

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Any chance of some tests? :-) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Ah yes, true. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > Your patch should also address possible overflow of the addition. Here there is no overflow. The patch limits prec of a little stronger (instead of 2147483647 to 2147483639 on a 32-bit platform). -- ___ Python t

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: > But it has to happen on any platform > with a 32-bit integer Not necessarily: it's undefined behaviour, so the compiler can do as it wishes. Your patch should also address possible overflow of the addition. -- _

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > Serhiy, what platform are you on? 32-bit Linux (Ubuntu), gcc 4.6. But it has to happen on any platform with a 32-bit integer (for 64-bit use 9223372036854775808). 214748364*10/10 == 214748364 -- test passed 214748364*10 + ('8'-'0') == -2147483648 -- oops!

[issue14700] Integer overflow in classic string formatting

Serhiy Storchaka added the comment: > Serhiy: FYI we use the versions field to indicate which versions the fix will > be made in, not which versions the bug occurs in. Since only 2.7, 3.2, and > 3.3 get bug fixes, I've changed the versions field to be just those three. > (3.1 and 2.6 are st

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: See get_integer in Objects/stringlib/unicode_format.h for a better way to do this sort of thing. -- ___ Python tracker ___ ___

[issue14700] Integer overflow in classic string formatting

Mark Dickinson added the comment: Indeed, Objects/unicodeobject.c (default branch) has this, at around line 13839: if ((prec*10) / 10 != prec) { PyErr_SetString(PyExc_ValueError, "prec too big");

[issue14700] Integer overflow in classic string formatting

R. David Murray added the comment: Serhiy: FYI we use the versions field to indicate which versions the fix will be made in, not which versions the bug occurs in. Since only 2.7, 3.2, and 3.3 get bug fixes, I've changed the versions field to be just those three. (3.1 and 2.6 are still in th

[issue14700] Integer overflow in classic string formatting

New submission from Serhiy Storchaka : Check for integer overflow for width and precision is buggy. Just a few examples (on platform with 32-bit int): >>> '%.21d' % 123 '00123' >>> '%.2147483648d' % 123 '123' >>> '%.2147483650d' % 123 Traceback (most recent call last): File ""