#26306: Memory leak in cached template loader
---------------------------------+------------------------------------
Reporter: AlexHill | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 1.9
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by AlexHill):
Thanks for your comments!
I wrote a detailed comment here last night, lost it due to not being
logged in in this tab, then ragequit in frustration.
The gist is: even with the leak plugged, 1.9 can cause significantly
higher memory usage than the same project using 1.8, because template
debug information is always cached. We can fix it by not caching debug
information when template debugging is off.
In 1.8, the thing that is cached when a template isn't found is the
TemplateDoesNotExist class itself. When the debug view wants to know which
paths were searched while looking for the template, it consults the
rendering engine again. In 1.9, that debug information is stored on an
exception instance, and those exceptions are cached.
I use Mezzanine, which tries a lot of different templates for each content
page, in order to allow you to override templates in various ways.
Significantly, it tries a template matching the exact slug of each page,
so that page templates can be individually overridden. In my case, I have
about 100,000 such pages, of which only a handful are overridden, so the
template cache is mostly full of "template missing" errors.
With app directory template discovery, 100,000 pages and about 50 items in
INSTALLED_APPS, I end up with ~5m template paths being cached, along with
their wrapping `Origin` objects, tuples, etc. Within a day or so this eats
most of the memory on my server.
I think it makes sense to have this information in the exception, but it's
debugging information and doesn't need to be stored when template
debugging is available. I propose restoring the old caching behaviour when
template debugging is off so that the overhead per missing template is
simply one entry in a dict.
--
Ticket URL: <https://code.djangoproject.com/ticket/26306#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.4e36c0218e73f61ed8936ea5723ec493%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
