On Jan 25, 2016 2:04 AM, "Frank Millman" <fr...@chagford.com> wrote: > > "Ian Kelly" wrote in message news:calwzidngogpx+cpmvba8vpefuq4-bwmvs0gz3shb0owzi0b...@mail.gmail.com... >> >> This seems to be a common misapprehension about asyncio programming. >> While coroutines are the focus of the library, they're based on >> futures, and so by working at a slightly lower level you can also >> handle them as such. So while this would be the typical way to use >> run_in_executor: >> >> async def my_coroutine(stuff): >> value = await get_event_loop().run_in_executor(None, >> blocking_function, stuff) >> result = await do_something_else_with(value) >> return result >> >> This is also a perfectly valid way to use it: >> >> def normal_function(stuff): >> loop = get_event_loop() >> coro = loop.run_in_executor(None, blocking_function, stuff) >> task = loop.create_task(coro) >> task.add_done_callback(do_something_else) >> return task > > > I am struggling to get my head around this. > > 1. In the second function, AFAICT coro is already a future. Why is it necessary to turn it into a task? In fact when I tried that in my testing, I got an assertion error - > > File: "C:\Python35\lib\asyncio\base_events.py", line 211, in create_task > task = tasks.Task(coro, loop=self) > File: "C:\Python35\lib\asyncio\tasks.py", line 70, in __init__ > assert coroutines.iscoroutine(coro), repr(coro) > AssertionError: <Future pending ... >
I didn't test this; it was based on the documentation, which says that run_in_executor is a coroutine. Looking at the source, it's actually a function that returns a future, so this may be a documentation bug. There's no need to get a task specifically. We just need a future so that callbacks can be added, so if the result of run_in_executor is already a future then the create_task call is unnecessary. To be safe, you could replace that call with asyncio.ensure_future, which accepts any awaitable and returns a future. > 2. In the first function, calling 'run_in_executor' unblocks the main loop so that it can continue with other tasks, but the function itself is suspended until the blocking function returns. In the second function, I cannot see how the function gets suspended. It looks as if the blocking function will run in the background, and the main function will continue. Correct. It's not a coroutine, so it has no facility for being suspended and resumed; it can only block or return. That's why the callback is necessary to schedule additional code to run after blocking_function finishes. normal_function itself can continue to make other non-blocking calls such as scheduling additional tasks, but it shouldn't do anything that depends on the result of blocking_function since it can't be assumed to be available yet. > I would like to experiment with this further, but I would need to see the broader context - IOW see the 'caller' of normal_function(), and see what it does with the return value. The caller of normal_function can do anything it wants with the return value, including adding additional callbacks or just discarding it. The caller could be a coroutine or another normal non-blocking function. If it's a coroutine, then it can await the future, but it doesn't need to unless it wants to do something with the result. Depending on what the future represents, it might also be considered internal to normal_function, in which case it shouldn't be returned at all. -- https://mail.python.org/mailman/listinfo/python-list
