#23830: Make expired session clearing on database backend scale better
-------------------------------------+-------------------------------------
Reporter: adamchainz | Owner:
Type: | adamchainz
Cleanup/optimization | Status: closed
Component: contrib.sessions | Version: master
Severity: Normal | Resolution:
Keywords: | worksforme
Has patch: 1 | Triage Stage:
Needs tests: 0 | Unreviewed
Easy pickings: 1 | Needs documentation: 0
| Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by adamchainz):
I can confirm there were no problems with signal handlers or foreign keys
making the fast deletion path ignored. The problem (at my previous
employer's, Memrise) was literally the DELETE FROM WHERE query touching
too many rows in one operation and locking the index[es] whilst it removes
them. The session table was seeing relatively high writes (memcached fleet
for reads) as there were a number of features mutating session data.
Actually at one point clearsessions made the site go down briefly. The
daily cron task had not run for a couple weeks due to misconfiguration, so
when it did run the DELETE query was removing around half the table. This
is all in MySQL but I'm sure postgres would behave similarly with such a
large table.
If you don't want to introduce the chunked deletion into django core I
understand, this problem must only occur with such high traffic sites. I'm
using more tools like pt-archiver ( http://www.percona.com/doc/percona-
toolkit/2.1/pt-archiver.html ) for such cleanup operations now anyway. At
least I had my first go at submitting a django patch though, thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/23830#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/068.f7060f8d2cab0cc75c2c51a5d140518d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
