[issue34277] EmailPolicy not followed
Change by Bryce Drennan : Removed file: https://bugs.python.org/file47720/test_header_folding.py ___ Python tracker <https://bugs.python.org/issue34277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34277] EmailPolicy not followed
New submission from Bryce Drennan : Starting in 3.6.4, the header flattening ignores the EmailyPolicy.utf8 attribute if a header is longer than maxlen. I believe this was introduced in https://github.com/python/cpython/pull/4693. Specifically this part: https://github.com/miss-islington/cpython/blob/8085ac188785ad0301760869a08b83c2945257a4/Lib/email/_header_value_parser.py#L2668-L2673 This causes problems as the dkim-signature header of parsed email messages gets mangled when they are flattened. It should look like this: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1532918961; bh=AwLeVe/FpXHJ9+VNy8QKwz2N5wuNF5ZkyXE3tLVBrFY=; h=Date:From:Reply-To:To:Subject:References:From:Subject; b=rSWZ7vyWIZqflUJS9ysVQvDxeoMxepEqPr/EoVkqpilCP1ryvci6/jCsFe75M2Jr5NJjzg6yJ6Xew8rpq8SMnZeNhTMmCK8jy\r\n WwSamcZ14t0LUZEt30+9Ump0KbPq+WRQK2rM9NnBVhE6pyvANfgsKMqgXlYzAmHk7P8cZ7ztJMSrtOeOr3u5RRNwvYJ+OYHZSFHiQZrPopNDKovVBcAc+6yVBI3YsI1qsgDmoQ/F5NszOLsBit2IkcvWr7z [...] but instead gets output like this: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048;\r\n t=1532918961; bh=AwLeVe/FpXHJ9+VNy8QKwz2N5wuNF5ZkyXE3tLVBrFY=;\r\n h=Date:From:Reply-To:To:Subject:References:From:Subject; =?utf-8?q?b=3DrSWZ?=\r\n =?utf-8?q?7vyWIZqflUJS9ysVQvDxeoMxepEqPr/EoVkqpilCP1ryvci6/jCsFe75M2Jr5NJjz?=\r\n =?utf-8?q?g6yJ6Xew8rpq8SMnZeN [...] Attached is a test that passes in 3.6.3 and fails in 3.6.4. -- components: email files: test_header_folding.py messages: 322653 nosy: barry, bryced, r.david.murray priority: normal severity: normal status: open title: EmailPolicy not followed versions: Python 3.6, Python 3.7 Added file: https://bugs.python.org/file47720/test_header_folding.py ___ Python tracker <https://bugs.python.org/issue34277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34277] EmailPolicy not followed
Bryce Drennan added the comment: Yes, utf8 is set to false. Despite that, the dkim-signature header, which contains no unicode characters, is getting filled with ?utf-8?q? values. My reading of the documentation of the utf8 flag is that headers should not be encoded like this if its set to False. Its possible I am misunderstanding. -- ___ Python tracker <https://bugs.python.org/issue34277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34277] EmailPolicy not followed
Bryce Drennan added the comment: That makes sense. Apologies for my misreading. Thanks for taking time to explain that. I think there is still something strange here since it's unnecessarily using encoded words when it could just "fold" without them. My tests with gmail show that it accepts a multi-line dkim-signature headers but does not handle the encoded words syntax. While not python's job to maintain compatibility with gmail, I suspect many DKIM implementations don't expect encoded words syntax and thus this change could cause many email handling systems to break. I'll dig in more and open a separate ticket. Thank you again for your time. -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue34277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34277] EmailPolicy not followed
Bryce Drennan added the comment: As far as I can tell in my manual tests with gmail, extra whitespace is fine. The addition of `=?utf-8?q?` is what trips both gmail and the python dkim library up. I agree that the paths you propose are viable. For now my email projects will be pinned to 3.6.3. If I find time, the new ticket will perhaps focus just on being able to "fold" without the encoded-words syntax. -- ___ Python tracker <https://bugs.python.org/issue34277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34881] unnecessary encoded-words usage breaks DKIM signatures
New submission from Bryce Drennan :
Since Python 3.6.4 folding of unstructured headers uses the encoded words
syntax even if there are no special characters.
This makes DKIM-Signature headers that are unreadable to google's gmail
servers. It may be that encoded-words are not valid in this header. I don't
see them mentioned here: https://tools.ietf.org/html/rfc6376#page-8
Here is the smallest test case I could create to demonstrate the issue.
One solution would be to add DKIM-Signature to the HeaderRegistry but I'm not
yet expert enough to execute this. I went down that path for a few hours.
Didn't see a straight-forward way to disable encoded words.
Setting EmailPolicy(max_line_length=None) does output without encoded words but
I worry that will cause different incompatibility issues.
from email.headerregistry import HeaderRegistry
from email.policy import SMTP
def test_unstructured_encoded_word_folding():
header = HeaderRegistry()('DKIM-Signature', 'a' * 85)
folded = header.fold(policy=SMTP.clone(refold_source=None))
print(f'\nDKIM-Signature: {header}')
print(folded)
assert '=?utf-8?q?' not in folded
Output:
DKIM-Signature:
a
DKIM-Signature: =?utf-8?q?aa?=
=?utf-8?q?aaa?=
AssertionError()!
--
components: email
messages: 326943
nosy: barry, bryced, r.david.murray
priority: normal
severity: normal
status: open
title: unnecessary encoded-words usage breaks DKIM signatures
versions: Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue34881>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
