Martin Panter added the comment: I opened Issue 23231 about fixing iterencode() and iterdecode() in the general case. I added a patch to Issue 13881 to fix StreamWriter for zlib and bz2, and to fix StreamWriter.writelines() in general.
I am adding a patch here to clarify the StreamReader API and fix the StreamReader for the zlib-codec. Plan of other things to do: bz2 StreamReader: Should be implemented similar to the zlib patch above, after Issue 15955 is resolved and we have a max_length parameter to use. Or could be based on Bz2File now. hex decoder: Shouldn’t be too hard to hack a stateful IncrementalDecoder that saves the leftover digit if given an odd number of digits. Create a generic codecs._IncrementalStreamReader class that uses an IncrementalDecoder and buffers unread decoded data, similar to my _IncrementalStreamWriter for Issue 13881. base64 encoder: IncrementalEncoder could encode in base64.MAXBINSIZE chunks base64 decoder: IncrementalDecoder could strip non-alphabet characters using regular expressions, decode in multiples of four characters quopri encoder: would require new implementation or major refactoring of quopri module quopri decoder: check for incomplete trailing escape codes (=, =<single hex digit>, =\r) and newlines (\r) uu encoder: write header and trailer via uu module; encode using b2a_uu() uu decoder: factor out header parsing from uu module; buffer and decode line by line based on encoded length unicode-escape, raw-unicode-escape: Stateful decoding would probably require a new -Stateful() function at the C level, though it might be easy to build from the existing implementation. I suggest documenting that stateful decoding is not supported for the time being. utf-7: As Walter said, proper stateful codec is not supported by the C API, despite PyUnicode_DecodeUTF7Stateful(); doing so would probably require significant changes. I suggest documenting a warning for stateful mode (including with TextIOWrapper) about suboptimal encoding and unlimited data buffering for the time being. punycode, unicode_internal: According to test_codecs.py, these also don’t work in stateful mode. Not sure on the details though. ---------- keywords: +patch Added file: http://bugs.python.org/file37718/zlib-reader.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20132> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
