[issue14850] The inconsistency of codecs.charmap_decode

Changes by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___ ___

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: Fixed. Thank you for your answers, Martin. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue14850] The inconsistency of codecs.charmap_decode

Roundup Robot added the comment: New changeset 33a8ef498b1e by Serhiy Storchaka in branch '2.7': Issue #14850: Now a chamap decoder treates U+FFFE as "undefined mapping" http://hg.python.org/cpython/rev/33a8ef498b1e New changeset 13cd78a2a17b by Serhiy Storchaka in branch '3.2': Issue #14850: No

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: I no one objects I will commit this next year. -- ___ Python tracker ___ ___ Python-bugs-list mail

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Serhiy Storchaka : -- stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Antoine Pitrou : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: Does anyone have objections against the idea or the implementation of the patch? Please review. -- ___ Python tracker ___ __

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Serhiy Storchaka : -- components: +Unicode keywords: +needs review versions: +Python 3.4 ___ Python tracker ___ ___ Python-

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: Patch updated to resolve conflict with issue15379. Added tests. Added patches for 3.2 and 2.7. -- Added file: http://bugs.python.org/file27387/decode_charmap_fffe-3.3.patch Added file: http://bugs.python.org/file27388/decode_charmap_fffe-3.2.patch Add

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file25934/decode_charmap_fffe.patch ___ Python tracker ___ ___ Python-bugs-

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Antoine Pitrou : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: > So the answer to your last question is "yes". I hope that the answer to > your other questions follows from that Thank you, this is the answer to all my questions. I've prepared a patch to treat U+FFFE in general mapping as “undefined mapping”. > (strictl

[issue14850] The inconsistency of codecs.charmap_decode

Martin v. Löwis added the comment: > integers or 1-character strings? What about general mapping? Should > any of them have 0xFFFE or '\uFFFE' represent an undefined mapping? The documentation says that the parameter "can be a dictionary mapping byte or a unicode string, which is treated as a

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: > What is the question? U+FFFE also represents an undefined mapping in > string subclasses. What about classes that not subclassed string but ducktyped string by implementing all string method? What about list/tuple/array.array of integers or 1-character str

[issue14850] The inconsistency of codecs.charmap_decode

Martin v. Löwis added the comment: >> U+FFFE is documented as representing an undefined mapping, > > Yes, using U+FFFE for representing an undefined mapping in strings is > normal, the question was about string subclasses. What is the question? U+FFFE also represents an undefined mapping in st

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: > U+FFFE is documented as representing an undefined mapping, Yes, using U+FFFE for representing an undefined mapping in strings is normal, the question was about string subclasses. And if we will correct it for string subclasses, how far we go any further? Ho

[issue14850] The inconsistency of codecs.charmap_decode

Serhiy Storchaka added the comment: > What is the use case for passing a string subclass to charmap_decode? Or in > other words, how did you stumble upon the bug? I stumbled upon it, rewriting the charmap decoder (issue14874). Now charmap decoder processes the two cases -- a more effective ca

[issue14850] The inconsistency of codecs.charmap_decode

Martin v. Löwis added the comment: U+FFFE is documented as representing an undefined mapping, see http://docs.python.org/dev/c-api/unicode.html?highlight=charmap#PyUnicode_DecodeCharmap So the base string case is correct; the derived string implementation also needs to invoke the error handle

[issue14850] The inconsistency of codecs.charmap_decode

Éric Araujo added the comment: What is the use case for passing a string subclass to charmap_decode? Or in other words, how did you stumble upon the bug? -- nosy: +eric.araujo ___ Python tracker

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Terry J. Reedy : -- nosy: +doerwalter, lemburg ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue14850] The inconsistency of codecs.charmap_decode

Changes by Antoine Pitrou : -- nosy: +loewis ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue14850] The inconsistency of codecs.charmap_decode

New submission from Serhiy Storchaka : codecs.charmap_decode behaves differently with native and user string as decode table. >>> import codecs >>> print(ascii(codecs.charmap_decode(b'\x00', 'replace', '\uFFFE'))) ('\ufffd', 1) >>> class S(str): pass ... >>> print(ascii(codecs.charmap_decode(b