Rotwang wrote: > Here's something odd I've found with the tokenize module: tokenizing 'if > x:\n y' and then untokenizing the result adds '\\\n' to the end. > Attempting to tokenize the result again fails because of the backslash > continuation with nothing other than a newline after it. On the other > hand, if the original string ends with a newline then it works fine. Can > anyone explain why this happens?
> I'm using Python 3.4.3 on Windows 8. Copypasted from iPython: This looks like a bug... $ python3.4 -c 'import tokenize as t, io; print(t.untokenize(t.tokenize(io.BytesIO(b"if x:\n y").readline)))' b'if x:\n y\\\n' ...that is fixed now: $ python3.6 -c 'import tokenize as t, io; print(t.untokenize(t.tokenize(io.BytesIO(b"if x:\n y").readline)))' b'if x:\n y' A quick search brought up multiple bug reports, among them http://bugs.python.org/issue12691 -- https://mail.python.org/mailman/listinfo/python-list
