#29244: Passing dict to a Func subclass fails silently on hash step
------------------------------+--------------------------------------
Reporter: Rob Jauquet | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+--------------------------------------
Description changed by Rob Jauquet:
Old description:
> With this code:
>
> {{{#!python
> from django.contrib.postgres.search import SearchQuery, Value, Func
> from django.core.paginator import Paginator
> from django.db.models import F, TextField, DateTimeField, Models
> from django.utils import timezone
>
> class Entry(Model):
> text = TextField()
> published_on = DateTimeField(default=timezone.now())
>
> class Headline(Func):
> function = 'ts_headline'
> output_field = TextField()
>
> def __init__(self, text, query, options=None):
> extra = {}
> expressions = [text, query]
> if options:
> opt_str = ''
> for key, value in options.items():
> opt_str += f'{key}={value},'
> expressions.append(Value(opt_str[:-1]))
> super().__init__(*expressions, **extra)
>
> entries = Entry.objects.annotate(
> text_highlighted=Headline(
> F('text'),
> SearchQuery('house'),
> options={'HighlightAll': False},
> )
> ).order_by('-published_on')
>
> paginator = Paginator(entries, 20)
>
> # evaluates the queryset
> paginator.page(1)
> }}}
>
> Passing options by {{{dict}}} to the {{{Func}}} subclass causes the count
> step for pagination to fail silently in paginator.py line 85 [#link1 (1)]
> which is in turn caused by a silenced TypeError during the hash in
> expressions.py line 372 [#link2 (2)].
>
> Then, because of the fallback to {{{len}}} on paginator.py line 90
> [#link3 (3)] the entire queryset is evaluated instead of evaluating just
> the specified page (and in this specific case, calling ts_headline on
> every row instead of just the first page).
>
> The correct way of passing options using {{{**options}}} instead of
> {{{options=None}}} fixes this case, but the silent error and fallback to
> calling {{{len}}} on a queryset causes an unexpected expensive query when
> using pagination.
>
> [=#link1 (1)]
> https://github.com/django/django/blob/master/django/core/paginator.py#L85
> [=#link2 (2)]
> https://github.com/django/django/blob/master/django/db/models/expressions.py#L372
> [=#link3 (3)]
> https://github.com/django/django/blob/master/django/core/paginator.py#L90
New description:
With this code:
{{{#!python
from django.contrib.postgres.search import SearchQuery, Value, Func
from django.core.paginator import Paginator
from django.db.models import F, TextField, DateTimeField, Models
from django.utils import timezone
class Entry(Model):
text = TextField()
published_on = DateTimeField(default=timezone.now())
class Headline(Func):
function = 'ts_headline'
output_field = TextField()
def __init__(self, text, query, options=None):
extra = {}
expressions = [text, query]
if options:
opt_str = ''
for key, value in options.items():
opt_str += f'{key}={value},'
expressions.append(Value(opt_str[:-1]))
super().__init__(*expressions, **extra)
entries = Entry.objects.annotate(
text_highlighted=Headline(
F('text'),
SearchQuery('house'),
options={'HighlightAll': False},
)
).order_by('-published_on')
paginator = Paginator(entries, 20)
# evaluates the queryset
paginator.page(1)
}}}
Passing options by {{{dict}}} to the {{{Func}}} subclass causes the count
step for pagination to fail silently in paginator.py line 85 [#link1 (1)]
which is in turn caused by a silenced TypeError during the hash in
expressions.py line 372 [#link2 (2)].
Then, because of the fallback to {{{len}}} on paginator.py line 90 [#link3
(3)] the entire queryset is evaluated instead of evaluating just the
specified page (and in this specific case, calling ts_headline on every
row instead of just the first page).
The correct way of passing options using {{{**options}}} instead of
{{{options=None}}} fixes this case, but the silent error and fallback to
calling {{{len}}} on a queryset causes an unexpected expensive query when
using pagination.
[=#link1 (1)]
https://github.com/django/django/blob/281c0223b376d6fa1a11e0726d824ed35cfe7524/django/core/paginator.py#L85
[=#link2 (2)]
https://github.com/django/django/blob/281c0223b376d6fa1a11e0726d824ed35cfe7524/django/db/models/expressions.py#L372
[=#link3 (3)]
https://github.com/django/django/blob/281c0223b376d6fa1a11e0726d824ed35cfe7524/django/core/paginator.py#L90
--
--
Ticket URL: <https://code.djangoproject.com/ticket/29244#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.c38f261540c7cc91cb7633f77162641d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
