#28231: bulk_create: avoid iterating `objs` more than necessary when
`bulk_size` is
provided
-------------------------------------+-------------------------------------
Reporter: Nir | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Database | Version: 1.11
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When `bulk_size` provided in `bulk_create`, a user might assume (As I
myself have) that `objs` iterable will not be iterated more than
`bulk_size` times at once, and that no more than roughly `bulk_size` Model
objects reside in memory at any given time.
When using `bulk_create` for relatively big sets of objects provided by a
generator object, it would be prefered to avoid iterating over the entire
generator object. Moreover, if not iterating over the generator object is
deemed unnecessary or out-of-scope for django it would be prefered to make
a comment on said behavior in documentation.
I suggest two possible solutions:
1. Document this behavior (`bulk_create` converts passed `objs` iterator
to a list), or
2. Avoid doing so when `bulk_size` is given (or default is other than
`None`. i.e. sqlite).
I did not research the possibility of avoiding the list conversion, but if
that solution is accepted by the community I volunteer to investigate
further and claim this ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/28231>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/049.5d15c5e0bef7668158c1470cffb3420e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
