On Fri, Sep 15, 2017 at 07:26:27AM -0700, DG wrote: > After reading this thread and answers on SO (e.g. How to combine 2 or more > querysets in a Django view? > <https://stackoverflow.com/questions/431628/how-to-combine-2-or-more-querysets-in-a-django-view?rq=1>) > > I'm still not sure if this code is fully lazy with Django 1.11+ or if it is > going to load everything into memory: > > qs1 = Model_A.objects.filter(foofield='foo').order_by('created_at') > qs2 = Model_B.objects.filter(barfield='bar').order_by('created_at') > > paginator = Paginator(sorted(chain(qs1, qs2), > key=lambda i:i.created_at > ), > 50) > > > If the querysets each referred to 500k results, would this blow up?
Yes.* * Unless you have a very powerful application server that can hold a copy of your entire databse in memory, and efficiently process it. What would happen is that the sorted() builtin function consumes the entire iterator you pass in, which pulls all the matching results from qs1, and then qs2, which will build model instances for all of them, and sorts the combined list in memory. Then you'd be taking the result of that sorting routine, and pass that to the paginator, which would discard all but 50 items from the potentially huge list of model instances. I don't think there's a way to not do this with the sorting that is built-in in Python, because it's always eager. Cheers, Michal -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/20170918130933.GP8762%40koniiiik.org. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: Digital signature
