Marc-Andre Lemburg <m...@egenix.com> added the comment: STINNER Victor wrote: > > STINNER Victor <victor.stin...@haypocalc.com> added the comment: > >> Why is this needed ? > > Short answer: to support filesystem encoding different than utf-8. See #8611 > for a longer explanation. > > Example: > > $ pwd > /home/SHARE/SVN/py3ké > $ PYTHONFSENCODING=ascii ./python test_fs_encoding.py > Fatal Python error: Py_Initialize: can't initialize sys standard streams > UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position > 20: ordinal not in range(128) > Abandon > > My patch fixes this specific case and prepare the work for the complete fix > (support different *locale* encodings, see #8611 and #9425). > > -- > > Longer answer: Py_FilesystemDefaultEncoding is changed too late. Some modules > are already loaded, sys.executable is already set, etc. > Py_FilesystemDefaultEncoding is changed but modules filenames are decoded > with utf-8 and should be "redecoded". > > It is not possible to set Py_FilesystemDefaultEncoding before loading the > first module. initfsencoding() loads codecs and encodings modules to check > the codec name. sys.executable is also set before initfsencoding(). > > Read my other messages of this issue to get other reasons why the patch is > needed. I explained other possibilities (but they don't work).
Thanks for the explanation. So the only reason why you have to go through all those hoops is to * allow the complete set of Python supported encoding names for the PYTHONFSENCODING * make sure that the Py_FilesystemDefaultEncoding is set to the actual name of the codec as used by the system Given that the redecoding of the filenames is fragile, I'd suggest to drop the encoding name check and then setting the variable right at the start of Py_Initialize(). If the encoding defined in PYTHONFSENCODING turns out not to be defined, the module loader will complain later on during startup. To play extra safe, you might run get_codec_name() at the same point in startup as you have initfsencoding() now. If something failed to load, you won't even get there. If things loaded fine, then you have a chance to safely double-check at that point. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9630> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
