[issue1466065] base64 module ignores non-alphabet characters

Julian Berman added the comment: Created issue22088 to address not having fixed Py2 here. -- nosy: +Julian ___ Python tracker ___ __

[issue1466065] base64 module ignores non-alphabet characters

Lukasz Taczuk added the comment: Could someone update the docs for python 2.7.3? This ticket is marked as closed and b64decode still silently ignores non base64 chars, but the documentation (for 2.7.3) still states that while calling b64decode "A TypeError is raised if (...) or if there are no

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: Committed in r86414. -- nosy: -BreamoreBoy resolution: -> accepted stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: Here is an updated patch that addresses the concerns I noted. I modified the tests: given that I've changed the code to raise binascii.Error as discussed, we don't really care from an API point of view what the error text is, just that the error is raised i

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: If you could update it that would be great. -- ___ Python tracker ___ ___ Python-bugs-list maili

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: I agree that it does look like RFC 3548 implicitly denies the use of \r and \n. A few *very* simple tests also makes it look like ommitting them breaks nothing, but we'd need to add a test that includes them before applying the patch. Other than that, I agree w

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: Comments on patch: 1) if I'm reading the RFC correctly, to be validating strictly in compliance with the RFC \r and \n should also raise an error. Do you agree? 2) We've pretty much dropped the convention of adding history notes to the file itself. 3) The

[issue1466065] base64 module ignores non-alphabet characters

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue1466065] base64 module ignores non-alphabet characters

Mark Lawrence added the comment: Attached patches against 2.7 and 3.2 contain doc and unit test changes so can someone please review and commit if acceptable. -- nosy: +BreamoreBoy stage: unit test needed -> patch review versions: +Python 3.1 ___ Py

[issue1466065] base64 module ignores non-alphabet characters

Changes by Raymond Hettinger : -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mail

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a documentation/unit-test/solution patch that adds a validate=False parameter to the b64decode function of Lib/base64.py, which may be set to True to have invalid base64 content be rejected with a TypeError. Patch built against Python 2.7, r74261. Note:

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a documentation/unit-test/solution patch that adds a validate=False parameter to the b64decode function of Lib/base64.py, which may be set to True to have invalid base64 content be rejected with a TypeError. Patch built against Python 3.2, r74261. --

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14598/1466065[3.2].diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14597/1466065[2.7].diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: And if the flag defaults to the current behavior that should satisfy the backward compatiblity issues. -- ___ Python tracker ___ ___

[issue1466065] base64 module ignores non-alphabet characters

Antoine Pitrou added the comment: > If the documentation is clear, then all future application developers > will know to check for validity using a regular expression like > '^[A-Za-z0-9+/\r\n]+={0,2}$'. Any existing applications in which > validity matters should already have a similar workarou

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: It isn't written that only MIME may ignore such content. The key terms there are 'may' and 'explicitly states otherwise'. If the documentation is clear, then all future application developers will know to check for validity using a regular expression like '^[A-Za-

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: Hmm. But if the module is used outside of MIME (which it can be, and in fact is in the stdlib itself), then an error must be raised in order to comply with that RFC. So it sounds like we really ought to have that flag. And I was even wrong about the appropri

[issue1466065] base64 module ignores non-alphabet characters

Changes by Raymond Hettinger : -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mail

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a documentation patch indicating that the ignore-invalid-characters behaviour is intentional, citing relevant RFCs for support. Patch built against Python 3.2, r74261. -- Added file: http://bugs.python.org/file14598/1466065[3.2].diff ___

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a documentation patch indicating that the ignore-invalid-characters behaviour is intentional, citing relevant RFCs for support. Patch built against Python 2.7, r74261. -- Added file: http://bugs.python.org/file14597/1466065[2.7].diff ___

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: RFC 3548, referenced by the base64 module's docs, has a rather different statement on how invalid characters should be treated. >From 2.3 Interpretation of non-alphabet characters in encoded data: Implementations MUST reject the encoding if it contains characte

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: It turns out that the RFC is unambiguous on this point. Quoting the base64 section of RFC 2045: The encoded output stream must be represented in lines of no more than 76 characters each. All line breaks or other characters not found in Table 1 must

[issue1466065] base64 module ignores non-alphabet characters

Guido van Rossum added the comment: It strikes me as simply a documentation bug. Maybe whoever wrote the docs just assumed it would work that way. I expect that changing it to insist on valid input would break some use cases. If you want a validating API, perhaps a new API could be added that va

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: I'll hold off on uploading new patches until someone makes a decision, then. It seems like, perhaps, simply amending the documentation would be sufficient, since this behaviour shouldn't break any valid messages that might reach this module. At worst, it'll just t

[issue1466065] base64 module ignores non-alphabet characters

Antoine Pitrou added the comment: Perhaps Guido remembers why the decision was made. -- nosy: +gvanrossum, pitrou versions: +Python 2.7, Python 3.2 -Python 2.6, Python 3.0 ___ Python tracker

[issue1466065] base64 module ignores non-alphabet characters

Amaury Forgeot d'Arc added the comment: I've dig into the history of the file and found this change: http://svn.python.org/view?view=rev&revision=13939 "- Illegal padding is now ignored. (Recommendation by GvR.)" The motivation at the time was based on "the general Internet philosophy": http:/

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14589/1466065[3.2-pure-python].diff ___ Python tracker ___ ___ Python-bugs

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14588/1466065[2.7-pure-python].diff ___ Python tracker ___ ___ Python-bugs

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14587/1466065[3.2].diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue1466065] base64 module ignores non-alphabet characters

Changes by Neil Tallim : Removed file: http://bugs.python.org/file14586/1466065[2.7].diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: Amaury is probably better qualified to answer that question, but I would think the C code version is preferable. -- ___ Python tracker ___ _

[issue1466065] base64 module ignores non-alphabet characters

Amaury Forgeot d'Arc added the comment: > Therefore, '!' should fail with a TypeError Here is your test case! "Errors should never pass silently." -- ___ Python tracker ___ __

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: R. David Murray, should I update the patches for both the pure-Python solution and the C solution, or is one domain preferable here? The Python-based solution keeps all of the invalid-character TypeErrors collected in the same module, but the C-based solution allow

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: According to the documentation cited by Seo Sanghyeon in the first post, "A TypeError is raised if s were incorrectly padded or if there are non-alphabet characters present in the string." The valid range of characters is [A-Za-z0-9], and one or two '='s may appea

[issue1466065] base64 module ignores non-alphabet characters

Amaury Forgeot d'Arc added the comment: What is the correct behavior for something like this? base64.b64decode('!') 2.6 silently returns ''. -- ___ Python tracker ___ _

[issue1466065] base64 module ignores non-alphabet characters

R. David Murray added the comment: If "it may be noisy where it was silent before", then add one of those cases and make sure the noise doesn't happen before your fix, and does happen after. If you have to check the value of the exception string for other tests, then do so. There are plenty of

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: I can't add a test for that without changing unrelated behaviour in the library that is already known to work fine or checking the string value of the raised exception, which seems like a bad idea, even though it would work. If a character is ignored and this lead

[issue1466065] base64 module ignores non-alphabet characters

Amaury Forgeot d'Arc added the comment: The new unit tests pass without modifying the library. Could you include a case that fails with the current version? -- nosy: +amaury.forgeotdarc ___ Python tracker __

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a patch that modifies Lib/base64.py to raise a TypeError on any unrecognized base64 input, rather than continuing silently. Also includes unit tests. Patch built against Python 3.2, r74246. Correction to previous text: base64 continues silently, discardi

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a patch that modifies Lib/base64.py to raise a TypeError on any unrecognized base64 input, rather than failing silently. Also includes unit tests. Patch built against Python 2.7, r74246. -- Added file: http://bugs.python.org/file14588/1466065[2.7

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a patch that modifies Modules/binascii.c to raise a TypeError on any unrecognized base64 input, rather than failing silently. Also includes unit tests. Patch built against Python 3.2, r74246. TypeError was preserved over ValueError, despite the lack of r

[issue1466065] base64 module ignores non-alphabet characters

Neil Tallim added the comment: Attached a patch that modifies Modules/binascii.c to raise a TypeError on any unrecognized base64 input, rather than failing silently. Also includes unit tests. Patch built against Python 2.7, r74246. TypeError was preserved over ValueError, despite the lack of r

[issue1466065] base64 module ignores non-alphabet characters

Changes by Daniel Diniz : -- stage: -> test needed type: -> behavior versions: +Python 2.6, Python 3.0 ___ Python tracker ___ ___