On Tue, 5 Jan 2021 at 13:44, Jean-Paul Calderone <exar...@twistedmatrix.com> wrote:
> On Tue, Jan 5, 2021 at 6:49 AM Barry Scott <barry.sc...@forcepoint.com> > wrote: > >> On Monday, 4 January 2021 04:01:42 GMT Ian Haywood wrote: >> > In investigating async file I/O I came across this. In a nutshell it's >> > the new epoll() >> >> > It's marginally more efficient although this is only apparent at very >> > high loads. >> >> > What's more interesting is that io_uring accepts files as >> > well as network/pipe handles: avoiding the need for threads. >> >> What threads? Why do you call out file FDs different from socket FDs? >> >> The point of io_uring is to avoid transitions between user and kernel >> right? >> Nothing to do with thread. >> >> In current twisted you can run complex network code without threads >> already. >> >> > Here's a good intro: https://unixism.net/loti/index.html >> >> Also there is full coverage of io_uring on lwn.net. >> Its a fast evolving kernel API. >> >> > If people think an IoUringReactor is worthwhile I'll open a ticket and >> > make a start. >> >> I'm guessing that you will need to write a Python extension to get at >> io_uring or use ctypes. Is that what you where expecting? >> >> > However it will need a reviewer... :-) >> >> Yes this is going to be complex code that few people have any experience >> with. >> > > Just so there's a counter-perspective out there, I would like to suggest > that reactors are neither magical nor particular complex in the grand > scheme of software and that if you start with the assumption that a piece > of code is going to be complex and hard to review then you will almost > certainly create a piece of software that is complex and hard to review. > > My recommendation would be to instead start with the assumption that it's > just a bit more mundane code binding a relatively small and straightforward > C API related to copying bytes from one location to another. Yea, maybe > there will be some hassles getting it to compile smoothly in all the > desired environments or maybe there will be a few tricky areas for memory > management or some other kind of low-level, localized bookkeeping - but > those kinds of issues don't have to make for a complex piece of code. > Starting from this assumption, try to produce an implementation that is > straightforward, well-factored, and easily reviewed. What do you have to > lose? > > Jean-Paul > > Hi, There is this wrapper https://pypi.org/project/liburing/ I can give it a try and review a possible implementation. My suggestion is to find a real world / production use case for the new reactor so that we can run more than unit / functional tests. -- Adi Roiban
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
