"Frank Millman" wrote in message news:n9c4p3$gmp$1...@ger.gmane.org...
Some of you may have been following my attempts to modify my asyncio app
so that it does not block when accessing the database. Here is an update.
Here is an update to my update ...
I came up with what felt like a good idea. Run the database handler in a
separate thread, pass requests to it using a queue.Queue, and get it to
pass results back using an asyncio.Queue.
It works, but I had a vague sense that performance was a bit sluggish, so
I tried the 'recommended' approach of using asyncio.run_in_executor() to
execute database calls in a separate thread. It felt a bit faster.
Now I have written a proper timing test, and the recommended approach is
much faster. I am not 100% sure of the reason, but I think the problem is
that, with my method, when the database tries to 'put' a row on the return
queue, it has to use 'loop.call_soon_threadsafe()', and this seems to
create a bottleneck.
I have come up with a plan that seems to provide a solution.
Instead of 'putting' one row at a time, let the database handler build up a
block of rows, and then 'put' the block.
I tried a block of 10, and it ran a lot faster. I increased it to 50, and it
ran faster again. I tried 100 and there was not much improvement, so 50
seems like an optimum number. The speed is now only slightly slower than
run_in_executor(), and it is more truly asynchronous.
Frank
--
https://mail.python.org/mailman/listinfo/python-list
