"Frank Millman" wrote in message news:n9hjfp$ad7$1...@ger.gmane.org...
However, my concern is not to maximise database performance, but to ensure
that in an asynchronous environment, one task does not block the others
from responding. My tests simulate a number of tasks running concurrently
and trying to access the database. Among other measurements, I track the
time that each database access commences. As I expected, tasks run with
'run_in_executor' run sequentially, i.e. the next one only starts when the
previous one has finished. This is not because the tasks themselves are
sequential, but because 'fetchall()' is (I think) a blocking operation.
Conversely, with my approach, all the tasks start within a short time of
each other. Because I can process the rows as they are received, it seems
to give each task a fairer time allocation. Not to mention that there are
very likely to be other non-database tasks running concurrently, and they
should also be more responsive.
It would be quite difficult to simulate all of this, so I confess that I
am relying on gut instinct at the moment.
It seems that my gut instinct was correct.
Up to now my timing tests have been run independently of my app, but now I
have embedded them so that I can run the tests while I am logged in as a
user.
I run a task every 10 seconds that runs 25 concurrent tasks, each reading a
database table of about 2000 rows.
Using run_in_executor() and cur.fetchall(), I experience delays of up to 2
seconds while the task is active.
Using my approach, the maximum I saw was about a tenth of a second, and that
is because I was looking for it - a normal user would not notice. Obviously
the task took longer, but I can live with that trade-off.
As I mentioned before, I could be using run_in_executor() in a naïve way,
and there could be better approaches. But until someone points out a better
way, I have nothing else to go on.
Frank
--
https://mail.python.org/mailman/listinfo/python-list
