#29229: QuerySet.values_list() combined with .extra() or .annotate() may produce
wrong .union()
-------------------------------------+-------------------------------------
Reporter: master | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: union values_list | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Regression introduced in 2.0, still present in 2.0.3.
>
> Easy context to reproduce the problem: suppose a Message model, in a
> postman app. Only one ordinary field is enough for the demo.
>
> {{{#!python
> qs1 = Message.objects.extra(select={'count': 0}).values_list('id',
> 'count').order_by()
> print(qs1.query) # as expected:
> # SELECT (0) AS "count", "postman_message"."id"
> # FROM "postman_message"
>
> qs2 =
> Message.objects.values('somefield').annotate(count=models.Count('pk')).annotate(id=models.Max('pk')).values_list('id',
> 'count').order_by()
> print(qs2.query) # as expected:
> # SELECT COUNT("postman_message"."id") AS "count",
> MAX("postman_message"."id") AS "id"
> # FROM "postman_message" GROUP BY "postman_message"."somefield"
>
> print(qs1.union(qs2).query) # !! WRONG !! the qs2 part is truncated:
> # SELECT (0) AS "count", "postman_message"."id" FROM "postman_message"
> # UNION
> # SELECT MAX("postman_message"."id") AS "id" FROM "postman_message" GROUP
> BY "postman_message"."somefield"
> }}}
>
> Compared to version 1.11, it comes from this addition in
> db/models/sql/compiler.py/get_combinator_sql():
> {{{#!python
> if not compiler.query.values_select and self.query.values_select:
> compiler.query.set_values(self.query.values_select)
> }}}
> Here, self.query.values_select is ('id',).
> For qs2, values_select is indeed empty, as coded in
> db/models/sql/query.py/set_values(),
> because in this case the two values_list() arguments are managed in
> annotation_names[].
New description:
Regression introduced in 2.0, still present in 2.0.3 [Edit: and backported
in 1.11.8].
Easy context to reproduce the problem: suppose a Message model, in a
postman app. Only one ordinary field is enough for the demo.
{{{#!python
qs1 = Message.objects.extra(select={'count': 0}).values_list('id',
'count').order_by()
print(qs1.query) # as expected:
# SELECT (0) AS "count", "postman_message"."id"
# FROM "postman_message"
qs2 =
Message.objects.values('somefield').annotate(count=models.Count('pk')).annotate(id=models.Max('pk')).values_list('id',
'count').order_by()
print(qs2.query) # as expected:
# SELECT COUNT("postman_message"."id") AS "count",
MAX("postman_message"."id") AS "id"
# FROM "postman_message" GROUP BY "postman_message"."somefield"
print(qs1.union(qs2).query) # !! WRONG !! the qs2 part is truncated:
# SELECT (0) AS "count", "postman_message"."id" FROM "postman_message"
# UNION
# SELECT MAX("postman_message"."id") AS "id" FROM "postman_message" GROUP
BY "postman_message"."somefield"
}}}
Compared to version 1.11 [Edit: I run 1.11.7 precisely], it comes from
this addition in db/models/sql/compiler.py/get_combinator_sql():
{{{#!python
if not compiler.query.values_select and self.query.values_select:
compiler.query.set_values(self.query.values_select)
}}}
Here, self.query.values_select is ('id',).
For qs2, values_select is indeed empty, as coded in
db/models/sql/query.py/set_values(),
because in this case the two values_list() arguments are managed in
annotation_names[].
--
Comment (by master):
Precisions about versions.
--
Ticket URL: <https://code.djangoproject.com/ticket/29229#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.b779afb5add0ac0de72b34af8c0afa99%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
