> On Nov 23, 2014, at 6:30 PM, Monty Taylor <mord...@inaugust.com> wrote: > > On 11/23/2014 06:13 PM, Robert Collins wrote: >> On 24 November 2014 at 11:01, victor stinner >> <victor.stin...@enovance.com> wrote: >>> Hi, >>> >>> I'm happy to announce you that I just finished the last piece of the puzzle >>> to add support for trollius coroutines in Oslo Messaging! See my two >>> changes: >>> >>> * Add a new aiogreen executor: >>> https://review.openstack.org/#/c/136653/ >>> * Add an optional executor callback to dispatcher: >>> https://review.openstack.org/#/c/136652/ >>> >>> Related projects: >>> >>> * asyncio is an event loop which is now part of Python 3.4: >>> http://docs.python.org/dev/library/asyncio.html >>> * trollius is the port of the new asyncio module to Python 2: >>> http://trollius.readthedocs.org/ >>> * aiogreen implements the asyncio API on top of eventlet: >>> http://aiogreen.readthedocs.org/ >>> >>> For the long story and the full history of my work on asyncio in OpenStack >>> since one year, read: >>> http://aiogreen.readthedocs.org/openstack.html >>> >>> The last piece of the puzzle is the new aiogreen project that I released a >>> few days ago. aiogreen is well integrated and fully compatible with >>> eventlet, it can be used in OpenStack without having to modify code. It is >>> almost fully based on trollius, it just has a small glue to reuse eventlet >>> event loop (get read/write notifications of file descriptors). >>> >>> In the past, I tried to use the greenio project, which also implements the >>> asyncio API, but it didn't fit well with eventlet. That's why I wrote a new >>> project. >>> >>> Supporting trollius coroutines in Oslo Messaging is just the first part of >>> the global project. Here is my full plan to replace eventlet with asyncio. >> >> ... >> >> So - the technical bits of the plan sound fine. >> >> On WSGI - if we're in an asyncio world, I don't think WSGI has any >> relevance today - it has no async programming model. While is has >> incremental apis and supports generators, thats not close enough to >> the same thing: so we're going to have to port our glue code to >> whatever container we end up with. As you know I'm pushing on a revamp >> of WSGI right now, and I'd be delighted to help put together a >> WSGI-for-asyncio PEP, but I think its best thought of as a separate >> thing to WSGI per se. It might be a profile of WSGI2 though, since >> there is quite some interest in truely async models. >> >> However I've a bigger picture concern. OpenStack only relatively >> recently switched away from an explicit async model (Twisted) to >> eventlet. >> >> I'm worried that this is switching back to something we switched away >> from (in that Twisted and asyncio have much more in common than either >> Twisted and eventlet w/magic, or asyncio and eventlet w/magic). >> >> If Twisted was unacceptable to the community, what makes asyncio >> acceptable? [Note, I don't really understand why Twisted was moved >> away from, since our problem domain is such a great fit for reactor >> style programming - lots of networking, lots of calling of processes >> that may take some time to complete their work, and occasional DB >> calls [which are equally problematic in eventlet and in >> asyncio/Twisted]. So I'm not arguing against the move, I'm just >> concerned that doing it without addressing whatever the underlying >> thing was, will fail - and I'm also concerned that it will surprise >> folk - since there doesn't seem to be a cross project blueprint >> talking about this fairly fundamental shift in programming model. > > I'm not going to comment on the pros and cons - I think we all know I'm > a fan of threads. But I have been around a while, so - for those who > haven't been: > > When we started the project, nova used twisted and swift used eventlet. > As we've consistently endeavored to not have multiple frameworks, we > entered in to the project's first big flame war: > > "twisted vs. eventlet" > > It was _real_ fun, I promise. But a the heart was a question of whether > we were going to rewrite swift in twisted or rewrite nova in eventlet. > > The main 'winning' answer came down to twisted being very opaque for new > devs - while it's very powerful for experienced devs, we decided to opt > for eventlet which does not scare new devs with a completely different > programming model. (reactors and deferreds and whatnot) > > Now, I wouldn't say we _just_ ported from Twisted, I think we finished > that work about 4 years ago. :) >
For whatever it’s worth, I find explicit async io to be _way_ easier to understand for the same reason I find threaded code to be a rats nest. The co-routine style of asyncio (or Twisted’s inlineCallbacks) solves almost all of the problems that I think most people have with explicit asyncio (namely the callback hell) while still getting the benefits. Glyph wrote a good post that mirrors my opinions on implicit vs explicit here: https://glyph.twistedmatrix.com/2014/02/unyielding.html. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
