Alex Gaynor wrote:
>> I recently had a proces hog about 1.8 GB RAM when looping through a
>> queryset with approx. 350k items as:
>>
>> for obj in Model.objects.all():
>> do_something(obj)
>>
>> I rewrote it to:
>>
>> objs = Model.objects.all().values_list("id", flat=True)
>> for obj_id in objs:
>> obj = Model.object.get(pk=obj_id)
>> do_something(obj)
>>
>> ... and my RAM usage was below 30 MB at all time.
>>
>
> You also executed 350k SQL queries. A better idea would be to start with:
Well, that in itself shouldn't cause that much RAM usage - just longer
execution time.
My guess, without looking at the queryset implementation, is that they
cache earlier, passed by items.
> for obj in Model.objects.all().iterator():
> do_something(obj)
Thank you. I didn't know about that function. That is certainly prettier
than my "hack" :)
I assumed that was the default behaviour when iterating. There shouldn't
be any need to cache previous items, as there (to my knowledge).is no
way to retrieve previous items from a python iterator.
Regards,
--
Christian Joergensen
http://www.technobabble.dk
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
