#26063: Regression in Django 1.9: SQLite can no longer handle more than 2000
values
in a "foo__in" filter
----------------------------------------------+--------------------
Reporter: rhertzog | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.9
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
Commit 4f6a7663bcddffb114f2647f9928cbf1fdd8e4b5 introduced a regression in
Django 1.9 because you can no longer have a queryset of the form {{{
Model.objects.filter(foo__in=array) }}} with more than 2000 items in
{{{array}}}.
The above changes generates a “SELECT QUOTE(?), ..., QUOTE(?)” query with
as many values as items in the array. This will hit the SQLite limit
SQLITE_MAX_COLUMN which defaults to 2000 and can only be increased up to
32767.
Before this change we were only limited by SQLITE_MAX_VARIABLE_NUMBER
which is lower by default (999) but which can be increased to a much
higher value. In Debian for example we use a value of 250000.
While the latter limit was unreasonably low (it just costs a bit of memory
and you can expect to have many parameters), the former limit make much
more sense IMO and I don't expect distributions to override the upstream
value.
I'm not sure what's the best way forward. Possibly process parameters by
batch of 2000.
(This bug has been initially reported in https://bugs.debian.org/809211 )
--
Ticket URL: <https://code.djangoproject.com/ticket/26063>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/051.1c1ce8d386cffc0052854803ce1caa60%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
