New submission from Serhiy Storchaka: Here is a file which is accepted by Python interpreter but is rejected by the tokenize module.
$ ./python nonutf8_coding_line.py $ ./python -m tokenize nonutf8_coding_line.py nonutf8_coding_line.py: error: invalid or missing encoding declaration for 'nonutf8_coding_line.py' Python itself checks that a line is UTF-8 encoded only if not found a magic comment. The tokenize module checks it before searching (issue14629). ---------- components: Interpreter Core, Library (Lib) files: nonutf8_coding_line.py messages: 197169 nosy: Arfrever, benjamin.peterson, brett.cannon, eric.snow, loewis, serhiy.storchaka priority: normal severity: normal status: open title: Non-UTF8 encoding line type: behavior versions: Python 3.3, Python 3.4 Added file: http://bugs.python.org/file31648/nonutf8_coding_line.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18961> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
