Am 13.03.21 um 01:01 schrieb Ian Haywood:
On 12/03/2021 8:29 pm, Tobias Oberstein wrote:
since with io_uring the queue/ring to append to never blocks for
writes, that buffering in twisted would lead to double buffering (as
the ring already buffers)
then, io_uring follows a completion IO model (as IOCP). quite
different from the "ready to write/read" model (as in epoll etc).
I am wondering, do you already have a design for these aspects of
io_uring / twisted?
I haven't thought about it in detail, but python presents mmap as
bytearray, so I would have thought it fairly easy to write directly to
the ring from Protocol.write
fwiw, couple of thoughts:
to make it "zero copy" right up into python, a couple of things would be
needed (there might be other ways of course ..):
the memory to write into on the producer side must be allocated and
managed via liburing
that memory might for example actually reside inside a NIC and be DMA'ed
into main memory ("zero CPU copy") or even directly mapped into the user
level address space ("true zero copy")
even when taking the very basic case of just transmitting an unstructed
bunch of bytes, to make it truly zero copy up into python, sth like
https://zero-buffer.readthedocs.io
might be needed rather than standard python str/bytes.
then, for structured data, if zero-buffer would be used in python lib of
flatbuffers for str/bytes
https://google.github.io/flatbuffers/
this would result in a truly zero-copy transfer _and_ serialization for
native structured objects.
all of that with zero/minimal syscalls.
in a way, the holy grail;)
cheers,
/Tobias
alternatives for flatbuffers:
https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
