#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom Forbes | Owner: Tom
Type: | Forbes
Cleanup/optimization | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tom Forbes):
I've been taking a look at this today. It's been a while since I've poked
around inside the expressions API, but I can see a couple of issues:
1. We generate a _lot_ of allocations during the whole expression
resolution process. 40% of the time is spent allocating according to
vmprof
2. "resolve_expression" for the Case statement is very copy happy. Each
resolution copies all the inner nodes, which then copy themselves.
3. We apparently needlessly call "resolve_expression" *twice* for each
bulk update. On the first invocation we call [it from this
method](https://github.com/django/django/blob/3259983f569151232d8e3b0c3d0de3a858c2b265/django/db/models/sql/subqueries.py#L105-L109)
which replaces the Q objects with WhereNodes. We then resolve *again*
during execution [from this with
statement](https://github.com/django/django/blob/master/django/db/models/query.py#L769-L770).
I'll keep investigating, but after naively disabling the first resolution
locally everything worked as expected, which implies for the happy path
this might be wasted.
--
Ticket URL: <https://code.djangoproject.com/ticket/31202#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/061.fcf46d1fd1f568545b9259319eba60aa%40djangoproject.com.
