#31973: TypeError loading data in JSONField if DB has native JSON support
-----------------------------------------+------------------------
Reporter: Adam Alton | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
Following today's security release, I've upgraded from Django 3.1 to 3.1.1
with no other code changes, and suddenly I'm getting this error when
loading a model instance from the DB:
{{{
list(MyModel.objects.all())
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py",
line 287, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py",
line 1303, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py",
line 70, in __iter__
for row in compiler.results_iter(results):
File "/usr/local/lib/python3.8/site-
packages/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/json.py", line 74, in from_db_value
return json.loads(value, cls=self.decoder)
File "/usr/local/lib/python3.8/json/__init__.py", line 341, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not list
}}}
From looking at the recent changes between 3.0 and 3.1, I'm pretty sure
that the problem was introduced in this commit:
https://github.com/django/django/commit/0be51d2226fce030ac9ca840535a524f41e9832c
Specifically, in django/db/models/fields/json.py, the removal of this from
`from_db_value`:
{{{
if connection.features.has_native_json_field and self.decoder is None:
return value
}}}
I'm pretty sure that this means that if the value has already been decoded
and the DB has got native JSON support, then it continues through this
function, attempting to decode the already-decoded data, and therefore
gets a `TypeError`. But note that the exception catching around
`json.loads` only catches `json.JSONDecodeError`, not `TypeError`, and
hence it dies.
I will see if I can write a test to prove this (or run the existing tests
with postgres, which has native JSON support). But I'm reporting this now,
as it seems like a fairly serious breaking change in 3.1.1. And maybe
someone else who understands the changes in the mentioned commit can fix
this faster than I can.
--
Ticket URL: <https://code.djangoproject.com/ticket/31973>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/052.94a77a346cb73ab24381e674a5444c28%40djangoproject.com.
