#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom | Owner: nobody
Forbes |
Type: | Status: new
Cleanup/optimization |
Component: Database | Version: master
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
A user has reported seeing extremely slow update times when using
bulk_update. With the django-bulk-update package, which does not use the
expressions API and constructs raw SQL directly, an update with 100,000
objects and and 10 fields takes 24 seconds. With the built in bulk_update
it takes 2 minutes and 24 seconds - 6x as slow.
The user has provided a reproduction case here: https://github.com/mikicz
/bulk-update-tests/blob/master/apps/something/models.py and
https://github.com/mikicz/bulk-update-
tests/blob/master/apps/something/test_bulk_update.py
From an initial look at the profiling that has been provided
(https://github.com/aykut/django-bulk-
update/files/4060369/inbuilt_pyinstrument.txt) it seems a lot of overhead
comes from building the SQL query rather than executing it - I think if we
can improve the performance it could speed up other workloads.
See: https://github.com/aykut/django-bulk-
update/issues/75#issuecomment-576886385
--
Ticket URL: <https://code.djangoproject.com/ticket/31202>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/046.738d868bb05ce5357dc4512f171988e0%40djangoproject.com.
