[issue27020] os.writev() does not accept generators (as buffers argument)

Марк Коренберг added the comment: Moreover, sendmsg() flattens iterable to a list -- this is not very good for performance and memory. Why not to fill IOV one-by-one, doubling it's size (via realloc) when need ? -- ___ Python tracker

[issue27020] os.writev() does not accept generators (as buffers argument)

Марк Коренберг added the comment: Proof for sendmsg: In [1]: import socket In [2]: (r, w) = socket.socketpair() In [3]: w.sendmsg((i for i in [b'1', b'2'])) Out[3]: 2 -- ___ Python tracker

[issue27020] os.writev() does not accept generators (as buffers argument)

Марк Коренберг added the comment: How sequence is iterated in writev(): https://github.com/python/cpython/blob/e4945dbd854befd04fe2d5ad99ab52c8d13dc162/Modules/posixmodule.c#L8086 How IOV is filled in semdmsg(): https://github.com/python/cpython/blob/fa5d369fc83f695d19eb57048efa7ec07ca3c1d7/Mod

[issue27020] os.writev() does not accept generators (as buffers argument)

Марк Коренберг added the comment: Pull request to asyncio: https://github.com/python/asyncio/pull/339 -- ___ Python tracker ___ ___ P

[issue27020] os.writev() does not accept generators (as buffers argument)

Марк Коренберг added the comment: Well, I'm improving asyncio performance. Instead of using bytearray as a big buffer and appending to it, I use deque of buffers. This eliminates memory copying. When writing is possible, I call writev()/sendmsg(). Unfortunatelly, OS sets limit on count of buf

[issue27020] os.writev() does not accept generators (as buffers argument)

Serhiy Storchaka added the comment: This makes sense. The purpose of writev() is to write all data in single atomic operation. In any case a sequence should be built. This looks rather as intentional behavior than as a bug to me. -- nosy: +serhiy.storchaka

[issue27020] os.writev() does not accept generators (as buffers argument)

New submission from Марк Коренберг: Unlike socket.sendmsg(), os.writev() does not support generators. Proof: In [4]: os.writev(1, [b'aa', b'bb', b'\n']) aabb Out[4]: 5 In [5]: os.writev(1, (i for i in [b'aa', b'bb', b'\n'])) ... TypeError: writev() arg 2 must be a sequence -- componen