#26063: Regression in Django 1.9: SQLite can no longer handle more than 2000
values
in a "foo__in" filter
-------------------------------------+-------------------------------------
Reporter: rhertzog | Owner: aaugustin
Type: Bug | Status: assigned
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by aaugustin):
Let's assume a statement with C columns and P parameters. With the current
implementation, on SQLite, when `DEBUG = True`:
(1) The statement itself is executed: C columns, P parameters. This
requires `C ≤ SQLITE_MAX_COLUMN` and `P ≤ SQLITE_MAX_VARIABLE_NUMBER`.
(2) Another statement is executed to escape the parameters: P columns, P
parameters. This requires `P ≤ MIN(SQLITE_MAX_COLUMN,
SQLITE_MAX_VARIABLE_NUMBER)`
The default values of SQLite are `SQLITE_MAX_COLUMN = 2000 >
SQLITE_MAX_VARIABLE_NUMBER = 999`. With these values,
`MIN(SQLITE_MAX_COLUMN, SQLITE_MAX_VARIABLE_NUMBER) =
SQLITE_MAX_VARIABLE_NUMBER`. As a consequence, any statement that will go
through (1) without raising an exception will also go through (2) without
raising an exception.
In your example, C = 1 and P = 2000. This will fail at step (1) with
SQLite's default values. It will reproduce the original reporter's
problem, but only on platforms where `SQLITE_MAX_VARIABLE_NUMBER` has been
increased to over 2000.
We need the test to pass or be skipped on all platforms, though.
I see two ways to achieve this:
- detecting the values of `SQLITE_MAX_COLUMN` and
`SQLITE_MAX_VARIABLE_NUMBER` and ajusting accordingly — which doesn't seem
possible in Python
- writing a test just for (2), without going through (1) — which I did.
--
Ticket URL: <https://code.djangoproject.com/ticket/26063#comment:14>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.5b0343c462269ff9a3dbc663a02e6b9a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
