Gareth Rees <g...@garethrees.org> added the comment:
I think I can make these changes independently and issue two patches, one
fixing the problems with untokenize listed here, and another improving tokenize.
I've just noticed a third bug in untokenize: in full mode, it doesn't handle
backslash-continued lines correctly.
Python 3.3.0a0 (default:c099ba0a278e, Aug 2 2011, 12:35:03)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on
darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from io import BytesIO
>>> from tokenize import tokenize, untokenize
>>> untokenize(tokenize(BytesIO('1 and \\\n not
2'.encode('utf8')).readline))
b'1 andnot 2'
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue12691>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
