#26318: Unexpected / duplicated queries on nested Prefetch queryset with
repeated
model
-------------------------------------+-------------------------------------
Reporter: aleontiev | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: master
(models, ORM) | Keywords: queryset
Severity: Normal | prefetch_related duplicate
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
We discovered an issue in {{{django.db.models.query.Prefetch}}} logic that
results in duplicate queries made when the leaves of certain prefetch
trees are accessed.
With these models:
{{{
from django.db import models
class Publisher(models.Model):
name = models.CharField(max_length=128)
class Author(models.Model):
name = models.CharField(max_length=128)
publishers = models.ManyToManyField('Publisher',
related_name='authors')
class Book(models.Model):
name = models.CharField(max_length=128)
publisher = models.ForeignKey('Publisher', related_name='books')
}}}
The following test fails:
{{{
from django.db.models.query import Prefetch
from django.test import TestCase
from .models import Author, Book, Publisher
def flatten(ls):
return list([i for s in ls for i in s])
class PrefetchTestCase(TestCase):
def setUp(self):
publisher = Publisher.objects.create(name='Publisher0')
Book.objects.create(name='Book0', publisher=publisher)
author = Author.objects.create(name='Author0')
author.publishers.add(publisher)
def test_prefetch_nested(self):
publishers = Publisher.objects.prefetch_related(
Prefetch(
'books',
Book.objects.all().prefetch_related(
Prefetch(
'publisher',
Publisher.objects.all().prefetch_related('authors')
)
)
)
)
with self.assertNumQueries(4):
publishers = list(publishers)
with self.assertNumQueries(0):
books = flatten([p.books.all() for p in publishers])
with self.assertNumQueries(0):
publishers = [b.publisher for b in books]
with self.assertNumQueries(0):
authors = flatten([p.authors.all() for p in publishers])
}}}
For more details (comments, queries executed) and an analogous green test-
case that uses the flat prefetch form to prefetch the same tree, see the
attached test package.
To run the tests:
{{{
tar -zxvf prefetch-bug-test.tar.gz
cd prefetch-bug-test
make test
}}}
This issue seemed very similar to
https://code.djangoproject.com/ticket/25546, but unfortunately the patch
for that ticket
(https://code.djangoproject.com/changeset/bdbe50a491ca41e7d4ebace47bfe8abe50a58211)
did not fix this problem.
Tested in Django 1.8, 1.9, and master.
--
Ticket URL: <https://code.djangoproject.com/ticket/26318>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/052.f26185adda9a5444973b704c6e551689%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
