> I agree that is counter to the basic philosophy, though there are
potential use cases where it may be unavoidable to use threads. It seems
the best solutions when it is then are to use run_in_executor, which
automatically handles the waking up of the loop.
While this doesn't provide an immediate solution, we're currently working
on implementing an asyncio.ThreadPool for Python 3.9, which is made to be
more self-contained (initialized and finalized within an async context
manager using ``async with``) and with a higher level API compared to
loop.run_in_executor(). In general, I'd recommend making use of either of
those two compared to directly using threads in asyncio.
That being said, if you need more fine-grained control, there's also one
other alternative low-level construct/design pattern you can use for
working with threads in asyncio that we use in the implementation of some
coroutine methods, such as one that I recently added to 3.9 called
loop.shutdown_default_executor(). Using a similar format to your above
example:
```
def _do_complete_something(fut, loop, ...):
try:
# perform some action, optionally get a result to set to the future
loop.call_soon_threadsafe(future.set_result, None) # replace None
with actual result if needed
except Exception as ex:
loop.call_soon_threadsafe(future.exception, ex)
async def compute_something(...):
# In general, I'd recommend using get_running_loop() to get the event
loop within a coroutine function.
# It has more predictable behavior compared to get_event_loop().
(Python 3.7+)
loop = asyncio.get_running_loop()
fut = loop.create_future()
th = threading.Thread(target=_do_compute_something, args=(fut,loop,
...))
try:
# use "return await future" if the value is desired
await future
finally:
th.join()
async def compute_stuff():
result = await compute_something(...)
asyncio.run(compute_stuff())
```
Hopefully that helps a bit, or at least gives you some some ideas.
On Thu, Feb 13, 2020 at 9:31 AM Brianvanderburg2 via Python-ideas <
[email protected]> wrote:
> I agree that is counter to the basic philosophy, though there are
> potential use cases where it may be unavoidable to use threads. It seems
> the best solutions when it is then are to use run_in_executor, which
> automatically handles the waking up of the loop. While the Queue object
> doesn't work with threads, though there does seem to be a library janus
> which provides a thread-capbable queue object which could be another
> solution.
>
>
> -----Original Message-----
> From: Guido van Rossum <[email protected]>
> To: Brian Allen Vanderburg II <[email protected]>
> Cc: Python-Ideas <[email protected]>
> Sent: Thu, Feb 13, 2020 1:18 am
> Subject: Re: [Python-ideas] Asyncio Future.set_result_threadsafe
>
> Sharing futures between threads like that goes counter to asyncio's basic
> philosophy (which is not to use threads :-).
>
> You already showed the solution:
> future._loop.call_soon_threadsafe(future.set_result, ...). If that's
> unacceptable for you, maybe you can wrap the future in a wrapper class that
> calls call_soon_threadsafe.
>
> On Wed, Feb 12, 2020 at 8:51 PM Brian Allen Vanderburg II via Python-ideas
> <[email protected]> wrote:
>
> Currently asyncio.futures.Future.set_result will result in any callbacks
> being scheduled using loop.call_soon instead of
> loop.call_soon_threadsafe. However in situations where the future's
> result is set from a different thread, the loop might not wake up as a
> result of this if it is currently sleeping.
>
>
> def compute_something(...):
> loop = asyncio.get_event_loop()
> future = loop.create_future()
> manager.run_computation(future, ...) # add the computation to a
> thread that is running it
> return future
>
> async def compute_stuff():
> result = await compute_something(...)
>
> loop = asyncio.get_event_loop()
> loop.run_until_complete(compute stuff())
> loop.close()
>
>
> The reasoning behind it is after yielding the Future object, the ready
> list of the event loop is empty so when waiting for selectors the
> timeout value is None and it just waits. When the other thread calls
> set_result, the loop ready list will get updated, but it's still waiting
> on selectors. The other thread could call
> future._loop.call_soon_threadsafe(future.set_result, ...), which writes
> some bytes to a socket in order to wake the even loop.
>
> I'm aware there are other ways to do things like this (queues,
> executors, etc). It just seems that, regardless of any method used, the
> code holding the future with the purpose of setting a result on it
> should be able to be blind to other implementation details.
> future.set_result should be all it needs to know to call even if from
> another thread, or if not, a method like future.set_result_threadsafe,
> which would hide the details of the loop.call_soon/call_soon_threadsafe
>
>
> _______________________________________________
> Python-ideas mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/PLKYKFWM4EM7DHUHWOBDU2JSJM57X7GM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
> _______________________________________________
> Python-ideas mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/CWV6UQUN4LFZHQK53L463OY4D4EROFXS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/ZNYGELT3I4L2G5JIQIEMFICKEGVQLXLG/
Code of Conduct: http://python.org/psf/codeofconduct/
