[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: On 05/13/2014 12:41 PM, Serhiy Storchaka wrote: > Note that this is not work with the punycode encoding (and may be some > third-party encodings). I do not understand. Could you elaborate? -- ___ Python tracker

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Serhiy Storchaka added the comment: Note that this is not work with the punycode encoding (and may be some third-party encodings). -- ___ Python tracker ___

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Antoine Pitrou added the comment: Thank you, I've committed tha patch to 3.5. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker _

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Roundup Robot added the comment: New changeset 2a1d63f09560 by Antoine Pitrou in branch 'default': Issue #21057: TextIOWrapper now allows the underlying binary stream's read() or read1() method to return an arbitrary bytes-like object (such as a memoryview). http://hg.python.org/cpython/rev/2a1d

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Changes by Nikolaus Rath : Added file: http://bugs.python.org/file35083/issue_21057_r3.diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Antoine Pitrou added the comment: > My usecase is that I have a binary stream class that internally uses > memoryviews. Ok, I think it is a reasonable use case. I'm gonna look at your patch and give it a review. -- stage: -> patch review ___ Pytho

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: My usecase is that I have a binary stream class that internally uses memoryviews. I would like to read text data from this stream and thus encapsulate it in a TextIOWrapper. Currently, TextIOWrapper (correctly) expects read() to return bytes and fails if it rec

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Martin v. Löwis added the comment: Nikolaus, can you please restate from scratch what the issue is? If the issue is still the one you posed in msg214776, I think the issue should be closed as "invalid" - it's *not* the case that there is no good reason for TextIOWrapper not accepting any bytes

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: Thanks for the feedback! I have attached an updated patch. I did not include any testcase because the patch did not create any new code paths, so I was assuming it'd be covered by the existing test case. But of course I was wrong. In the revised patch, I added

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Changes by Nikolaus Rath : Added file: http://bugs.python.org/file34646/issue21057.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Changes by Nikolaus Rath : Removed file: http://bugs.python.org/file34645/issue21057.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: I'm attaching a patch that enables TextIOWrapper to work with bytes-like objects from the underlying file descriptor. The code changes are pretty small, without introducing any significant additional complexity. For streams providing bytes objects, this patch

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: On 03/26/2014 03:43 AM, STINNER Victor wrote: >> class MyByteStream(BytesIO): >>def read1(self, len_): >>return memoryview(super().read(len_)) >> bs = MyByteStream(b'some data in ascii\n') > > I guess that you are trying to implement a zero-copy I/O.

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

STINNER Victor added the comment: > class MyByteStream(BytesIO): >def read1(self, len_): >return memoryview(super().read(len_)) > bs = MyByteStream(b'some data in ascii\n') I guess that you are trying to implement a zero-copy I/O. The problem is that BytesIO does copy data. Example:

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

STINNER Victor added the comment: 2014-03-26 0:35 GMT+01:00 Antoine Pitrou : > - they are immutable: you can keep an internal reference to a bytes object > and be sure it won't change under your feet Is it possible to request this feature using PyObject_GetBuffer()? I don't see such flag. If i

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Serhiy Storchaka added the comment: > If someone is willing to do the work (and I am), is there a reason *not* to > allow TextIOWrapper to accept bytes-like objects? Yes, there are. The code which works only with bytes is much simpler. Not only TextIOWrapper, but many other classes in the stdli

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: Yes, bytes objects have some advantages. But if a bytes object is desired, it can always be created from bytes-like object. If a BufferedIOBase instance is required to only provide bytes objects, this conversion is forced even when it may not be necessary. If

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Antoine Pitrou added the comment: bytes objects have two things going for them: - they have the full bytes API (all the startswith(), etc. methods) - not only buffer access - they are immutable: you can keep an internal reference to a bytes object and be sure it won't change under your feet Th

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: On 03/25/2014 01:39 PM, Serhiy Storchaka wrote: > read1() should return bytes. MyByteStream doesn't implement the > io.BufferedIOBase interface. Indeed, this is what this issue is about :-). The question is: is there a good reason to require io.BufferedIOBase i

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Serhiy Storchaka added the comment: read1() should return bytes. MyByteStream doesn't implement the io.BufferedIOBase interface. -- nosy: +serhiy.storchaka ___ Python tracker __

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

STINNER Victor added the comment: I'm interested to review a patch, but I'm not sure that read1() can return a type different than bytes. You can use the Py_buffer API for your patch. -- nosy: +haypo, pitrou versions: +Python 3.4 ___ Python tracker

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

Nikolaus Rath added the comment: If someone is willing to review, I'd be happy to write a patch for this. -- ___ Python tracker ___ __

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

New submission from Nikolaus Rath: In Python 3.4, TextIOWrapper can not read from streams that return bytearrays or memoryviews: from io import TextIOWrapper, BytesIO class MyByteStream(BytesIO): def read1(self, len_): return memoryview(super().read(len_)) bs = MyByteStream(b'some d