[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: I backported your last change to distutils2 as f5a74b1f9473. -- ___ Python tracker ___ ___ Python-bugs-

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: > I applied pkginfo_utf8.patch to Python 3.2 and 3.3. If you apply patches to distutils, please add tests for the fixed behavior. (Sorry if I wasn’t reactive on this one.) -- ___ Python tracker

[issue9561] distutils: set encoding to utf-8 for input and output files

Roundup Robot added the comment: New changeset 56ab3257ca13 by Victor Stinner in branch 'default': Issue #9561: packaging now writes egg-info files using UTF-8 http://hg.python.org/cpython/rev/56ab3257ca13 -- ___ Python tracker

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: I applied pkginfo_utf8.patch to Python 3.2 and 3.3. Python 2.7 is not affected, it does already encode Unicode to UTF-8. -- ___ Python tracker

[issue9561] distutils: set encoding to utf-8 for input and output files

Roundup Robot added the comment: New changeset fb4d2e6d393e by Victor Stinner in branch '3.2': Issue #9561: distutils now reads and writes egg-info files using UTF-8 http://hg.python.org/cpython/rev/fb4d2e6d393e New changeset 3c080bf75342 by Victor Stinner in branch 'default': Merge 3.2: Issue

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: > Ping. What's the progress on this? Will this ever be fixed? Some functions has been fixed in the new packaging module, but not in the distutils yet. -- ___ Python tracker _

[issue9561] distutils: set encoding to utf-8 for input and output files

Michał Górny added the comment: Ping. What's the progress on this? Will this ever be fixed? -- ___ Python tracker ___ ___ Python-bugs-

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: >> Okay. I guess you’ll use codecs.open in 2.7 > Oh, Python 2.7... DistributionMetadata of distutils encodes most > values to byte strings (get_xxx() methods calls self._encode_field). I forgot that. No change is needed in 2.7. > I checked, there is not bootstra

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: > Okay. I guess you’ll use codecs.open in 2.7 Oh, Python 2.7... DistributionMetadata of distutils encodes most values to byte strings (get_xxx() methods calls self._encode_field). It would be possible to use codecs.open(), but an Unicode file expects Unicode

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: > pkginfo_utf8.patch: distutils uses UTF-8 to write PKG-INFO and > .egg-info, instead of the locale encoding. It should be applied to > 2.7, 3.2 and 3.3. Okay. I guess you’ll use codecs.open in 2.7; please make sure there is no bootstrapping issue for the build

[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by STINNER Victor : Added file: http://bugs.python.org/file22524/packaging_pkginfo_utf8.patch ___ Python tracker ___ ___ Python-bugs-li

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: pkginfo_utf8.patch: distutils uses UTF-8 to write PKG-INFO and .egg-info, instead of the locale encoding. It should be applied to 2.7, 3.2 and 3.3. packaging_pkginfo_utf8.patch: packaging tests use UTF-8 to write PKG-INFO files, instead of the locale encoding

[issue9561] distutils: set encoding to utf-8 for input and output files

Arfrever Frehtes Taifersar Arahesis added the comment: Please file a separate issue. -- nosy: +Arfrever ___ Python tracker ___ ___ Pyt

[issue9561] distutils: set encoding to utf-8 for input and output files

Michał Górny added the comment: Now that installing scripts with unicode characters was fixed, shall I open a separate bug for writing egg files with utf8 chars in author name? -- ___ Python tracker __

[issue9561] distutils: set encoding to utf-8 for input and output files

Marc-Andre Lemburg added the comment: Éric Araujo wrote: > > Éric Araujo added the comment: > > Definitely. We can fix real bugs in distutils, but sometimes it’s best to > avoid disruptive changes and let distutils with its buggy behavior and let > the packaging module have the best behavi

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: Definitely. We can fix real bugs in distutils, but sometimes it’s best to avoid disruptive changes and let distutils with its buggy behavior and let the packaging module have the best behavior. -- versions: +Python 3.3 _

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: I started to patch packaging to fix this issue in the packaging module: issue #12112. We might leave distutils unchanged and improve the packaging module instead (because previous experiments proved that distutils should not be touched or it break random stuf

[issue9561] distutils: set encoding to utf-8 for input and output files

Hagen Fürstenau added the comment: Created issue 10419 for the encoding problem in "build_scripts". -- ___ Python tracker ___ ___ Pyth

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: This issue might be splitted in multiple issue: one issue per file type (eg. Makefile, RPM spec file, etc.). -- ___ Python tracker ___

[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by Mike Auty : -- nosy: +ikelos ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/m

[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by Michał Górny : -- nosy: +mgorny ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by Éric Araujo : -- versions: +3rd party ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue9561] distutils: set encoding to utf-8 for input and output files

Changes by Hagen Fürstenau : -- nosy: +hagen ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue9561] distutils: set encoding to utf-8 for input and output files

Toshio Kuratomi added the comment: >>> - RPM spec files, which use ASCII or UTF-8 according to >>> http://en.opensuse.org/openSUSE:Specfile_guidelines#Specfile_Encoding but >>> it’s not confirmed in >>> http://www.rpm.org/max-rpm/s1-rpm-build-creating-spec-file.html (linked >>> from the LSB site

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: Note that any change requires a test. -- assignee: tarek -> eric.araujo stage: -> patch review type: -> behavior versions: +Python 2.7, Python 3.1 ___ Python tracker ___

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: [Toshio, I made you nosy for a question about RPM .spec files] >> - PKG-INFO (METADATA in distutil2), that already uses a trick to support >> Unicode, but your change would replace it in a better way; > Which "trick"? Some values are explicitly allowed to use Uni

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: I attached a patch to #6011 to set the encoding to read the Makefile. -- ___ Python tracker ___ ___

[issue9561] distutils: set encoding to utf-8 for input and output files

STINNER Victor added the comment: > - PKG-INFO (METADATA in distutil2), that already uses a trick to support > Unicode, but your change would replace it in a better way; Which "trick"? > - MANIFEST, which with your fix would gain the ability to handle non-ASCII > paths, which is a feature or a

[issue9561] distutils: set encoding to utf-8 for input and output files

Éric Araujo added the comment: There are different kind of files created by write_file: - PKG-INFO (METADATA in distutil2), that already uses a trick to support Unicode, but your change would replace it in a better way; - MANIFEST, which with your fix would gain the ability to handle non-ASCI

[issue9561] distutils: set encoding to utf-8 for input and output files

New submission from STINNER Victor : While working on #9425 (support non-ascii characters in python directory name with ascii locale), I wrote a patch for distutils.file_util(): set encoding to utf-8 and errors to surrogateescape. See the patch with comments at: http://codereview.appspot.com/18