#16614: Support server-side cursors for queryset iteration in database backends
-------------------------------------+-------------------------------------
Reporter: toofishes | Owner:
Type: New feature | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: memory cursors | Triage Stage: Accepted
database |
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by keturn):
Okay, I've added
[https://github.com/keturn/django/commit/dd1c959eb817a7502c2e5f6405938301d05c277f
#diff-db5fb8fdfbba5c6ac7595dd636d9b558R93 a test], which is better than no
tests, but still leaves some to be desired. That is, this test makes sure
that a call to `chunked()` does not fail horribly, but it doesn't test the
salient difference between `chunked` and `iterator`, that `chunked` uses a
server-side cursor and doesn't load large result sets in to memory all at
once.
Other things that it seems like might be valuable to test:
* what if you have two `chunked()` queries at once, and alternate taking
objects from each of them, do they successfully not get in each other's
way?
* that may be the same as: if you took the `_named_cursor_idx` out of the
chunked_cursor name, what would break?
Considering the API, is it necessary to add a new `QuerySet.chunked()`, or
can we change `QuerySet.iterator()` to have this behaviour by default?
(because it's what I always ''assumed'' that `iterator` did until I ran
out of RAM.)
It does change the run-time characteristics to some degree, but I'm having
a hard time coming up with a situation where it would really break an
existing use case.
--
Ticket URL: <https://code.djangoproject.com/ticket/16614#comment:20>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/067.805390f55763221b4c3cb297c731e53d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
