#33355: Optimize SQLite backend connection functions
-------------------------------------+-------------------------------------
Reporter: Adam | Owner: nobody
Johnson |
Type: | Status: assigned
Cleanup/optimization |
Component: Database | Version: dev
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
The SQLite backend registers a bunch of data conversion functions when
creating a new connection, with `conn.create_function`. These functions
use the `@none_guard` decorator to check for `None`s in arguments:
{{{
def none_guard(func):
"""
Decorator that returns None if any of the arguments to the decorated
function are None. Many SQL functions return NULL if any of their
arguments
are NULL. This decorator simplifies the implementation of this for the
custom functions registered below.
"""
@functools.wraps(func)
def wrapper(*args, **kwargs):
return None if None in args else func(*args, **kwargs)
return wrapper
}}}
This decorator is wasteful. At call time, it makes Python coerce arguments
to *args and **kwargs and back , and as a decorator it forces two layers
of functions where one would do.
We can save this time by explicitly "inlining" the `None` checks. This
saves a small amount of time per call, but considering that these
functions can be called thousands of times in a single query, it adds up
fast. (...perhaps millions on larger data sets or suboptimal queries!)
Additionally, time and memory is wasted on every new SQLite connection
applying the decorator to create new copies of the same function. (And
`list_aggregate()` creates the exact same classes each time.) Considering
that Django creates a new connection per request by default (and per
test), this adds up.
A small benchmark with one of the shorter functions (Python 3.10.0):
{{{
In [1]: import functools
In [2]: def none_guard(func):
...: """
...: Decorator that returns None if any of the arguments to the
decorated
...: function are None. Many SQL functions return NULL if any of
their arguments
...: are NULL. This decorator simplifies the implementation of this
for the
...: custom functions registered below.
...: """
...: @functools.wraps(func)
...: def wrapper(*args, **kwargs):
...: return None if None in args else func(*args, **kwargs)
...: return wrapper
...:
In [3]: @none_guard
...: def _sqlite_rpad(text, length, fill_text):
...: return (text + fill_text * length)[:length]
...:
In [4]: def _sqlite_rpad_inlined(text, length, fill_text):
...: if text is None or length is None or fill_text is None:
...: return None
...: return (text + fill_text * length)[:length]
...:
In [5]: %timeit _sqlite_rpad("Hello world", 20, " ")
534 ns ± 30.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [6]: %timeit _sqlite_rpad_inlined("Hello world", 20, " ")
328 ns ± 26.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
}}}
~40% reduction.
--
Ticket URL: <https://code.djangoproject.com/ticket/33355>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/053.139c13ee93c003162fa48ab3ee07922e%40djangoproject.com.
