[issue9630] Reencode filenames when setting the filesystem encoding

Marc-Andre Lemburg added the comment: STINNER Victor wrote: > > STINNER Victor added the comment: > > Le mercredi 29 septembre 2010 13:45:15, vous avez écrit : >> Marc-Andre Lemburg added the comment: >> >> STINNER Victor wrote: >>> STINNER Victor added the comment: >>> >>> Forget my previo

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: I commited redecode_modules_path-4.patch as r85115 in Python 3.2. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: Le mercredi 29 septembre 2010 13:45:15, vous avez écrit : > Marc-Andre Lemburg added the comment: > > STINNER Victor wrote: > > STINNER Victor added the comment: > > > > Forget my previous message, I forgot important points. > > > >> So the only reason why

[issue9630] Reencode filenames when setting the filesystem encoding

Marc-Andre Lemburg added the comment: STINNER Victor wrote: > > STINNER Victor added the comment: > > Forget my previous message, I forgot important points. > >> So the only reason why you have to go through >> all those hoops is to >> >> * allow the complete set of Python supported encoding

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: Forget my previous message, I forgot important points. > So the only reason why you have to go through > all those hoops is to > > * allow the complete set of Python supported encoding > names for the PYTHONFSENCODING > > * make sure that the Py_FilesystemDef

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: Le vendredi 24 septembre 2010 14:35:29, Marc-Andre Lemburg a écrit : > Thanks for the explanation. So the only reason why you have to go through > all those hoops is to > > * allow the complete set of Python supported encoding names >for the PYTHONFSENCODI

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > Some things about your patch: > - as Amaury said, functions should be named "redecode*" > rather than "reencode*" Yes, as written before (msg117269), I will do it in my next patch. > - please use -1 for error return, not 1 Ok. > - have you tried to measur

[issue9630] Reencode filenames when setting the filesystem encoding

Marc-Andre Lemburg added the comment: STINNER Victor wrote: > > STINNER Victor added the comment: > >> Why is this needed ? > > Short answer: to support filesystem encoding different than utf-8. See #8611 > for a longer explanation. > > Example: > > $ pwd > /home/SHARE/SVN/py3ké > $ PYTHO

[issue9630] Reencode filenames when setting the filesystem encoding

Changes by Antoine Pitrou : -- nosy: +brett.cannon, loewis ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > Not sure it's related, but there seems to be a bug: It's not a bug, it's a feature :-) If you specify a non-existing locale, the GNU libc fails back to ascii. $ locale -a C français french fr_FR fr...@euro fr_FR.iso88591 fr_fr.iso885...@euro fr_FR.utf8 $ L

[issue9630] Reencode filenames when setting the filesystem encoding

Antoine Pitrou added the comment: Some things about your patch: - as Amaury said, functions should be named "redecode*" rather than "reencode*" - please use -1 for error return, not 1 - have you tried to measure if it made Python startup slower? -- _

[issue9630] Reencode filenames when setting the filesystem encoding

Antoine Pitrou added the comment: >It is not possible to set Py_FilesystemDefaultEncoding before loading >the first module. initfsencoding() loads codecs and encodings modules to >check the codec name. Not sure it's related, but there seems to be a bug: $ ./python -c "import sys; print(sys.g

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > Why is this needed ? Short answer: to support filesystem encoding different than utf-8. See #8611 for a longer explanation. Example: $ pwd /home/SHARE/SVN/py3ké $ PYTHONFSENCODING=ascii ./python test_fs_encoding.py Fatal Python error: Py_Initialize: can't

[issue9630] Reencode filenames when setting the filesystem encoding

Marc-Andre Lemburg added the comment: STINNER Victor wrote: > > STINNER Victor added the comment: > > New version of the patch: > - reencode sys.path_importer_cache (and remove the last FIXME) > - fix different reference leaks > - catch PyIter_Next() failures > - create a subfunction to r

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > I would rename the feature to something like "redecode-modules" Yes, right. I will rename the functions before commiting the patch. -- ___ Python tracker ___

[issue9630] Reencode filenames when setting the filesystem encoding

Changes by STINNER Victor : Removed file: http://bugs.python.org/file18561/reencode_modules_path-2.patch ___ Python tracker ___ ___ Python-bugs

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: New version of the patch: - reencode sys.path_importer_cache (and remove the last FIXME) - fix different reference leaks - catch PyIter_Next() failures - create a subfunction to reencode sys.modules: it's easier to review and manager errors in shorter funct

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > Do we really want to support this kind of configuration? There is also a problem is the directory name is b'py3k\xe9': at startup (utf-8 encoding), the name is decoded to 'py3k\udce9'. When the locale encoding is set to iso-8859-1: the name should be reenco

[issue9630] Reencode filenames when setting the filesystem encoding

Amaury Forgeot d'Arc added the comment: > Python is installed in a directory called b'py3k\xc3\xa9' > and your locale is C Do we really want to support this kind of configuration? -- ___ Python tracker

[issue9630] Reencode filenames when setting the filesystem encoding

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: Another possibility is to use _Py_char2wchar() + PyUnicode_FromWideChar() / _Py_wchar2char() + PyUnicode_AsWideChar() to decode / encode filenames. These functions use the locale encoding. This solution was possible in Python 3.1, but no more in Python 3.2 be

[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor added the comment: > since the modules were successfully imported, surely it means that > their filenames where correctly computed and encoded? So why is the > __filename__ attribute wrong? Python starts with 'utf-8' encoding. If the new encoding is "smaller" (unable to encode a

[issue9630] Reencode filenames when setting the filesystem encoding

Amaury Forgeot d'Arc added the comment: Some thoughts: since the modules were successfully imported, surely it means that their filenames where correctly computed and encoded? So why is the __filename__ attribute wrong? -- ___ Python tracker

[issue9630] Reencode filenames when setting the filesystem encoding

Amaury Forgeot d'Arc added the comment: I would rename the feature to something like "redecode-modules": the filenames were decoded with the wrong encoding, and must be decoded again. -- nosy: +amaury.forgeotdarc ___ Python tracker

[issue9630] Reencode filenames when setting the filesystem encoding

Changes by STINNER Victor : -- title: Reencode filenames of all module and code objects when setting the filesystem encoding -> Reencode filenames when setting the filesystem encoding ___ Python tracker ___