New submission from STINNER Victor <victor.stin...@haypocalc.com>:
If the script filename is not decodable from the filesystem encoding, Python
fails with a UnicodeEncodeError when we reach the recursion limit. The problem
doesn't come from the user script, but from Python internals. It is difficult
to understand and the user expects a
A longer explanation:
test_runpy fails with a pydebug build if the script filename is not encodable
to UTF-8. In pydebug build only, PyEval_EvalFrameEx() encodes the frame
filename to UTF-8. If the filename contains a surrogate character (which only
occurs on UNIX with undecodable filename),the encoding function fails.
PyEval_EvalFrameEx() ignores the error except if we hit the recusion limit (if
the overflowed attribute of the thread state if set): in this case, the error
is not ignored.
To reproduce the problem, change the Python directory (your local repository)
to an undecodable filename (eg. b'py3k\xe9\xff' with UTF-8 locale encoding) and
run: ./python Lib/test/regrtest.py test_py.
Solutions :
I propose to remove the filename variable from PyEval_EvalFrameEx() because it
is only used by the old gdb macros: python-gdb.py doesn't need it anymore.
=> see attached patch
Or if you really want to keep it, tstate->overflowed should be reinitialized.
But on overflow, other variables are changed, like _Py_CheckRecursionLimit. I
don't know this code enough to write a correct patch, but the minimal patch is:
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1234,7 +1234,7 @@ PyEval_EvalFrameEx(PyFrameObject *f, int throwflag)
filename = _PyUnicode_AsString(co->co_filename);
if (filename == NULL && tstate->overflowed) {
/* maximum recursion depth exceeded */
- goto exit_eval_frame;
+ tstate->overflowed = 0;
}
PyErr_Restore(error_type, error_value, error_traceback);
}
----------
components: Unicode
files: ceval_filename.patch
keywords: patch
messages: 128285
nosy: haypo
priority: normal
severity: normal
status: open
title: UnicodeEncodeError on recusion limit if the script filename is
undecodable
versions: Python 3.2, Python 3.3
Added file: http://bugs.python.org/file20728/ceval_filename.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue11168>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
