Feature Requests item #1326277, was opened at 2005-10-13 18:27 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1326277&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: paul rubin (phr) Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.count wraps around after maxint Initial Comment: See below. This goes against the notion of int/long unification and can cause weird unexpected behavior, almost like a buffer overflow. It should promote to a long at the appropriate time, or that's not feasible, then raise an exception, don't wrap around silently. Xrange is also not so great about this. It at least raises an exception if you give it too large an endpoint, but promoting to long would be better. Steven D'Aprano and others on clpy pointed this out. >>> from itertools import count >>> b=2**31 - 3 >>> c = count(b) >>> for i in range(5): ... print c.next() ... 2147483645 2147483646 2147483647 -2147483648 -2147483647 >>> ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-02-07 19:07 Message: Logged In: YES user_id=80475 Originator: NO Nows raises an OverflowError instead of silently wrapping around. See revisions 53665 and 53666. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2005-10-14 10:11 Message: Logged In: YES user_id=72053 You're right, the docs do describe that behavior of itertools.count (someone on clpy thought otherwise, IIRC). I don't see anything in http://docs.python.org/lib/built-in-funcs.html about what enumerate does. It looks my p3-750 can enumerate about 400k iterations of itertools.repeat(None) per second, so it can reach maxint in about 1.5 hours, but I don't feel like running it that long to see what happens. At minimum, enumerate's doc should be updated to say what it does. I suppose it's a matter of opinion but I'd take the view that both of these wraparounds (assuming enumerate wraps around) are bugs even if they're documented bugs. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-10-14 02:12 Message: Logged In: YES user_id=80475 It's not a bug -- it is the documented behavior. The simple work-around is to roll your own generators: def count(n): while 1: yield n n += 1 def enumerate(iterable): c = 0 for elem in iterable: yield (c, elem) c += 1 Will look at possibly enhancing this for Py2.5. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2005-10-14 00:53 Message: Logged In: YES user_id=72053 If both fixes are feasible then promoting to long is definitely the correct one. Raising an exception is just a kludge to at least not do something horrible silently. However, on a fast 32 bit machine, counting past 2**31 or something is quite realistic. A web server might send out that many packets in a few days or weeks. It shouldn't crash or go crazy after running for a long time and overflowing maxint. It occurs to me, the enumerate iterator should also be checked and fixed if needed. It, too, can overflow maxint if counting something like a network stream. Maybe there are some more iterators besides these, which need the same attention. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2005-10-14 00:18 Message: Logged In: YES user_id=33168 I agree something should be done. Raymond, which behaviour would you prefer? I can implement if you want (just let me know and assign back to me). BTW, I don't have the same problem. I need to set b = 2**63 - 3 :-) (using current CVS). ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2005-10-13 18:29 Message: Logged In: YES user_id=72053 I forgot to say, the test example is Python 2.4.1 on linux. I don't know if 2.4.2 has a fix. I don't see anything about it in the database. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1326277&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
