#34791: Issue when using Prefetch objects in prefetch_related
-------------------------------------+-------------------------------------
Reporter: Maxime Toussaint | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: prefetch, | Triage Stage:
prefetch_related | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Maxime Toussaint:
Old description:
> There seems to be an issue when using a Prefetch object to fetch
> something that has already been fetched. Here is an example: so let's say
> I have a Pizza and some Toppings. Now I want to get all the toppings, but
> for some reason I also want to separately fetch only the toppings that
> are out of stock. I could do something like:
>
> {{{
> queryset = Pizza.objects.all().prefetch_related(
> 'toppings',
> Prefetch('toppings',
> queryset=Topping.objects.filter(is_in_stock=False),
> to_attr='out_of_stock_toppings'),
> )
> }}}
>
> This looks good, but if we run it, it will fail, saying
> out_of_stock_toppings is not an attribute of Pizza. However, if I were to
> do it like this instead:
> {{{
> queryset = Pizza.objects.all().prefetch_related(
> Prefetch('toppings',
> queryset=Topping.objects.filter(is_in_stock=False),
> to_attr='out_of_stock_toppings'),
> 'toppings',
> )
> }}}
>
> then all works fine. Looking at the code, this seems to be because the
> name used by a field to validate the cache is not the name used to store
> the data on the model, but rather simply the name of the field, so it
> collides. I have not tested it, but I think in the second example, the
> data returned will actually be the filtered data, not the full expected
> queryset. Note that this is my first time reading that part of the code,
> so there could be things I missed.
>
> Now this is a bit of a nonsensical example, but when using rest_framework
> with serializers, this type of situation could come up, where one
> serializer needs it formatted a certain way and this issue could arise
> (it has for me).
>
> I am not sure what the best way to fix this would be, but I feel like
> setting a to_attr should make the cache take that new field name into
> account instead of the field name.
New description:
Note: Edited the description following the discussion
There seems to be an issue when using a Prefetch object to fetch something
that has already been fetched. The issue only seems to happen when there
is depth in the prefetch. Here is an example I made this morning that
fails:
{{{
pizzas = Pizza.objects.all().prefetch_related(
"toppings__origin",
Prefetch(
"toppings__origin",
queryset=Country.objects.filter(label="China"),
to_attr="china",
),
)
china = pizzas[0].toppings.all()[0].china
}}}
Here, when trying to get china, I would assume it to either be a Country
object or None. However, I get the message: AttributeError: 'Topping'
object has no attribute 'china'
Here are the models I set up for my test:
{{{
class Country(models.Model):
label = models.CharField(max_length=50)
class Pizza(models.Model):
label = models.CharField(max_length=50)
class Topping(models.Model):
pizza = models.ForeignKey(Pizza, on_delete=models.CASCADE,
related_name="toppings")
label = models.CharField(max_length=50)
origin = models.ForeignKey(
Country, on_delete=models.CASCADE, related_name="toppings"
)
}}}
And here are the queries made when calling the queryset:
{{{
1. SELECT "tests_pizza"."id", "tests_pizza"."label",
"tests_pizza"."provenance_id" FROM "tests_pizza"
2. SELECT "tests_topping"."id", "tests_topping"."pizza_id",
"tests_topping"."label", "tests_topping"."origin_id" FROM "tests_topping"
WHERE "tests_topping"."pizza_id" IN (1, 2)
3. SELECT "tests_country"."id", "tests_country"."label",
"tests_country"."continent_id" FROM "tests_country" WHERE
"tests_country"."id" IN (2, 3, 4)
}}}
Note that the filter by label='china' has completely disappeared.
Now, if I switch the prefetches around like so:
{{{
pizzas = Pizza.objects.all().prefetch_related(
Prefetch(
"toppings__origin",
queryset=Country.objects.filter(label="China"),
to_attr="china",
),
"toppings__origin",
)
}}}
Fetching china now works, and here are the queries being made:
{{{
1. SELECT "tests_pizza"."id", "tests_pizza"."label",
"tests_pizza"."provenance_id" FROM "tests_pizza"
2. SELECT "tests_topping"."id", "tests_topping"."pizza_id",
"tests_topping"."label", "tests_topping"."origin_id" FROM "tests_topping"
WHERE "tests_topping"."pizza_id" IN (1, 2)
3. SELECT "tests_country"."id", "tests_country"."label",
"tests_country"."continent_id" FROM "tests_country" WHERE
("tests_country"."label" = 'China' AND "tests_country"."id" IN (2, 3, 4))
4. SELECT "tests_country"."id", "tests_country"."label",
"tests_country"."continent_id" FROM "tests_country" WHERE
"tests_country"."id" IN (2, 3, 4)
}}}
This time, both calls to Country were made.
I did follow the code a bit yesterday, and I believe it comes from the
fact that the ForeignKey field defines the cache name as being simply the
name of the field. I am not certain though, and it would likely require
someone with more knowledge than me to look into it.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34791#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018a23f6151e-a5c5038b-b8ca-4e6d-b799-1c71264eaa5b-000000%40eu-central-1.amazonses.com.
