[issue5803] email/quoprimime: encode and decode are very slow on large messages

R. David Murray added the comment: I've reviewed this and applied it to both 3.3 and 3.4 in order to fix issue 20206. Thanks, Serhiy. -- stage: patch review -> committed/rejected status: open -> closed versions: +Python 3.3 ___ Python tracker

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Roundup Robot added the comment: New changeset 4c5b1932354b by R David Murray in branch '3.3': #20206, #5803: more efficient algorithm that doesn't truncate output. http://hg.python.org/cpython/rev/4c5b1932354b New changeset b6c3fc21286f by R David Murray in branch 'default': Merge #20206, #5803

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Serhiy Storchaka added the comment: Here is a patch for 3.4 based on Matt's patch with additional optimizations. It speeds up body_encode() and header_encode(). $ ./python -m timeit -s "from email.quoprimime import body_encode as encode; x = open('Lib/decimal.py').read()[:10]" "encode(x)"

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Serhiy Storchaka added the comment: 3.2+ already use StringIO, there is no nonlinearity. On 2.7 I couldn't find the nonlinearity with the texts of up to 100 MB. Therefore the path outdated as bugfix. However it shows 20-30x speedup on 2.7. It would be worth port it to 3.4. This will be a sign

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by R. David Murray : -- assignee: r.david.murray -> components: +email versions: -Python 3.1 ___ Python tracker ___ ___ Pytho

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by R. David Murray : -- versions: +Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by R. David Murray : -- assignee: -> r.david.murray nosy: +r.david.murray stage: needs patch -> patch review versions: +Python 3.3 ___ Python tracker ___

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Matt Cain added the comment: I re-wrote encode() to be simpler and faster. My version runs about 10 times faster on a 30KB message. I have tested it somewhat but not rigorously see attached patch -- keywords: +patch nosy: +cainmatt Added file: http://bugs.python.org/file20635/quoprimi

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Antoine Pitrou added the comment: Any news? -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Dave Baggett added the comment: Yes, sorry, I meant "built-in list type" not "array". Your point about using lists this way is valid, and is why I used array.array('c'). I will do as you suggest and try all three methods. I did time the array.array approach vs. the one currently in the code and

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Antoine Pitrou added the comment: The patch must be against the SVN trunk, in standard unified diff ("svn diff" does the trick). What do you call standard arrays? Do you mean the builtin list type? Storing one separate character per list element is a poor choice because it will waste a lot of m

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Dave Baggett added the comment: I can certainly generate a patch for you. What form would you like it in, and against what source tree? Also, do you have a preference between the use of array.array vs. standard arrays? (I don't know whether it's good or bad to depend on "import array" in quoprim

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by Antoine Pitrou : -- priority: -> normal stage: -> needs patch ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Antoine Pitrou added the comment: Do you have a patch? Also, detailed timing examples are welcome (you can e.g. use the "timeit" module from the command line). -- nosy: +pitrou versions: +Python 2.7, Python 3.1 -Python 2.6 ___ Python tracker

[issue5803] email/quoprimime: encode and decode are very slow on large messages

Changes by Dave Baggett : -- type: -> performance ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.p

[issue5803] email/quoprimime: encode and decode are very slow on large messages

New submission from Dave Baggett : The implementation of encode and decode are slow, and scale nonlinearly in the size of the message to be encoded/decoded. A simple fix is to use an array.array('c') and append to it, rather than using string concatenation. This change makes the code more than