On Mon, Mar 4, 2013 at 1:34 AM, Aymeric Augustin <[email protected]> wrote: > On 4 mars 2013, at 01:07, Shai Berger <[email protected]> wrote: ... >> The use of savepoints in Django apps so far has been very little, as far as >> I know. One point >> I'm unsure of is the interaction of savepoints with cursors, since querysets >> are lazy; so the scenario >> I'm worrirf about is, generally speaking, >> >> @atomic >> def main(): >> for obj in query_with_more_than_100_objects: >> try: >> handle(obj) >> except Bad: >> pass >> >> @atomic >> def handle(obj): >> if good(obj): >> mark_good(obj) >> obj.save() >> else: >> raise Bad(obj) >> >> Will the next (database) fetch after an exception is raised get the right >> objects? >> >> My reading of the Postgresql documentation is that it will do the right >> thing, not so sure about >> the other backends. > > Django currently doesn't support server side cursors — at least on > PostgreSQL, most likely on > other databases too. Accessing the first object loads the entire queryset > instantly.
Actually, the queryset API tries pretty hard to avoid this: https://github.com/django/django/blob/2cd0edaa477b327024e4007c8eaf46646dcd0f21/django/db/models/query.py#L954 It is true that by default, both psycopg and mysqldb do load the full resultset despite using the use of .fetchmany rather than .fetchall https://github.com/django/django/blob/0c82b1dfc48b4870e8fbcfb782ae02cdca821e1f/django/db/models/sql/compiler.py#L765 But this is not django's fault, and I have always wished more drivers would handle streaming results better. In any case, as far as I know, the above code (mixing 2 queries with partial result returns on a single connection) has never worked properly, and I think it will stay equally broken in the new world. I think this could be fairly easily (manually) tested by setting GET_ITERATOR_CHUNK_SIZE to an absurdly low size. Which is to say, +1 from me. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
