Drekin added the comment: Antoine Pitrou: I understand. It would be nice to have that new Python string based readline hook. Its default implementation could be to call PyOS_Readline and decode the bytes using sys.stdin.encoding (as the tokenizer currently does). Tokenizer then woudn't need to decode if it called the new hook.
Victor Stinner: I'm going to try the approach of reencoding my stream to UTF-8. So then my UTF-16-LE encoded stream is decoded, then encoded to UTF-8, interpreted as null-terminated *char, which is returned to the tokenizer, which again decodes it and encodes to UTF-8. I wonder if the last step could be short-circuited. What is this UTF8 flag to Python parser? I couldn't find any information. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue17620> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
