#27675: Django postgres JSONField encoding
-------------------------------------+-------------------------------------
Reporter: Hisham Waleed Karam | Owner: nobody
Type: Bug | Status: new
Component: contrib.postgres | Version: 1.10
Severity: Normal | Resolution:
Keywords: models JSONField | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by ErikW):
* status: closed => new
* resolution: needsinfo =>
Comment:
I have the same thing happening to me.
I have a model using a JSONField (from django.contrib.postgres.fields).
When I examine the field from an ipython shell (also happening when
interacting with the field from views), it is returning the contents as a
string instead of a dict (ie '{}'). I checked table in Postgres, and it
is in fact a jsonb column. Not sure why it's not deserializing the field.
If I save the model, it then escapes the field again which adds the
additional backslashes mentioned by Hisham above.
This behavior only started after upgrading this codebase from Django 1.9
to 1.10.5 -- the field worked correctly in 1.9.x.
{{{#!python
from django.contrib.postgres.fields import JSONField
class CRMOnboarding(models.Model):
data = JSONField(default=dict)
}}}
IPython Shell:
{{{#!python
In 13:58:31 [6]: x.crmonboarding_set.all().last()
Out 13:58:31 [6]: <CRMOnboarding: CRM -
bb69b886-25d5-4d59-a4d6-23bebc91af70>
In 13:58:34 [7]: crm=_
In 13:58:36 [8]: crm.data
Out 13:58:36 [8]: '{"fields": {"dob": "null", "vin": "null", "city":
"city", "email": "email", "phone": "phone", "state": "state", "Domain":
"null", "Handle": "null", "Status": "null", "zip_code": "zip_code",
"address_1": "address_1", "address_2": "address_2", "auto_fuel": "null",
"auto_make": "null", "auto_size": "null", "auto_type": "null",
"auto_year": "null", "last_name": "last_name", "auto_based": "null",
"auto_class": "null", "auto_model": "null", "auto_style": "null",
"first_name": "first_name", "Syntax good": "null", "ahh_num_cars": "null",
"Invalid Reason": "null", "auto_make_model": "null", "confidence_code":
"null", "Catch All Domain": "null", "auto_manufacturer": "null",
"auto_purchase_date": "null", "auto_purchase_type": "null"}}'
In 13:58:54 [9]: data=_
In 13:58:57 [10]: crm.save()
In 13:58:59 [11]: x.crmonboarding_set.all().last()
Out 13:58:59 [11]: <CRMOnboarding: CRM -
bb69b886-25d5-4d59-a4d6-23bebc91af70>
In 13:59:01 [12]: crm=_
In 13:59:02 [13]: crm.data
Out 13:59:02 [13]: '"{\\"fields\\": {\\"dob\\": \\"null\\", \\"vin\\":
\\"null\\", \\"city\\": \\"city\\", \\"email\\": \\"email\\", \\"phone\\":
\\"phone\\", \\"state\\": \\"state\\", \\"Domain\\": \\"null\\",
\\"Handle\\": \\"null\\", \\"Status\\": \\"null\\", \\"zip_code\\":
\\"zip_code\\", \\"address_1\\": \\"address_1\\", \\"address_2\\":
\\"address_2\\", \\"auto_fuel\\": \\"null\\", \\"auto_make\\": \\"null\\",
\\"auto_size\\": \\"null\\", \\"auto_type\\": \\"null\\", \\"auto_year\\":
\\"null\\", \\"last_name\\": \\"last_name\\", \\"auto_based\\":
\\"null\\", \\"auto_class\\": \\"null\\", \\"auto_model\\": \\"null\\",
\\"auto_style\\": \\"null\\", \\"first_name\\": \\"first_name\\",
\\"Syntax good\\": \\"null\\", \\"ahh_num_cars\\": \\"null\\", \\"Invalid
Reason\\": \\"null\\", \\"auto_make_model\\": \\"null\\",
\\"confidence_code\\": \\"null\\", \\"Catch All Domain\\": \\"null\\",
\\"auto_manufacturer\\": \\"null\\", \\"auto_purchase_date\\": \\"null\\",
\\"auto_purchase_type\\": \\"null\\"}}"'
In 13:59:26 [14]: import json
In 13:59:29 [15]: json.loads(data)
Out 13:59:29 [15]:
{u'fields': {u'Catch All Domain': u'null',
u'Domain': u'null',
u'Handle': u'null',
u'Invalid Reason': u'null',
u'Status': u'null',
u'Syntax good': u'null',
u'address_1': u'address_1',
u'address_2': u'address_2',
u'ahh_num_cars': u'null',
u'auto_based': u'null',
u'auto_class': u'null',
u'auto_fuel': u'null',
u'auto_make': u'null',
u'auto_make_model': u'null',
u'auto_manufacturer': u'null',
u'auto_model': u'null',
u'auto_purchase_date': u'null',
u'auto_purchase_type': u'null',
u'auto_size': u'null',
u'auto_style': u'null',
u'auto_type': u'null',
u'auto_year': u'null',
u'city': u'city',
u'confidence_code': u'null',
u'dob': u'null',
u'email': u'email',
u'first_name': u'first_name',
u'last_name': u'last_name',
u'phone': u'phone',
u'state': u'state',
u'vin': u'null',
u'zip_code': u'zip_code'}}
In 13:59:33 [16]: crm.data
Out 13:59:33 [16]: '"{\\"fields\\": {\\"dob\\": \\"null\\", \\"vin\\":
\\"null\\", \\"city\\": \\"city\\", \\"email\\": \\"email\\", \\"phone\\":
\\"phone\\", \\"state\\": \\"state\\", \\"Domain\\": \\"null\\",
\\"Handle\\": \\"null\\", \\"Status\\": \\"null\\", \\"zip_code\\":
\\"zip_code\\", \\"address_1\\": \\"address_1\\", \\"address_2\\":
\\"address_2\\", \\"auto_fuel\\": \\"null\\", \\"auto_make\\": \\"null\\",
\\"auto_size\\": \\"null\\", \\"auto_type\\": \\"null\\", \\"auto_year\\":
\\"null\\", \\"last_name\\": \\"last_name\\", \\"auto_based\\":
\\"null\\", \\"auto_class\\": \\"null\\", \\"auto_model\\": \\"null\\",
\\"auto_style\\": \\"null\\", \\"first_name\\": \\"first_name\\",
\\"Syntax good\\": \\"null\\", \\"ahh_num_cars\\": \\"null\\", \\"Invalid
Reason\\": \\"null\\", \\"auto_make_model\\": \\"null\\",
\\"confidence_code\\": \\"null\\", \\"Catch All Domain\\": \\"null\\",
\\"auto_manufacturer\\": \\"null\\", \\"auto_purchase_date\\": \\"null\\",
\\"auto_purchase_type\\": \\"null\\"}}"'
In 13:59:38 [17]: crm.data = json.loads(data)
In 13:59:42 [19]: crm.data
Out 13:59:42 [19]:
{u'fields': {u'Catch All Domain': u'null',
u'Domain': u'null',
u'Handle': u'null',
u'Invalid Reason': u'null',
u'Status': u'null',
u'Syntax good': u'null',
u'address_1': u'address_1',
u'address_2': u'address_2',
u'ahh_num_cars': u'null',
u'auto_based': u'null',
u'auto_class': u'null',
u'auto_fuel': u'null',
u'auto_make': u'null',
u'auto_make_model': u'null',
u'auto_manufacturer': u'null',
u'auto_model': u'null',
u'auto_purchase_date': u'null',
u'auto_purchase_type': u'null',
u'auto_size': u'null',
u'auto_style': u'null',
u'auto_type': u'null',
u'auto_year': u'null',
u'city': u'city',
u'confidence_code': u'null',
u'dob': u'null',
u'email': u'email',
u'first_name': u'first_name',
u'last_name': u'last_name',
u'phone': u'phone',
u'state': u'state',
u'vin': u'null',
u'zip_code': u'zip_code'}}
In 13:59:46 [20]: crm.save()
In 13:59:48 [21]: x.crmonboarding_set.all().last()
Out 13:59:48 [21]: <CRMOnboarding: CRM -
bb69b886-25d5-4d59-a4d6-23bebc91af70>
In 13:59:50 [22]: crm=_
In 13:59:52 [23]: crm.data
Out 13:59:52 [23]: '{"fields": {"dob": "null", "vin": "null", "city":
"city", "email": "email", "phone": "phone", "state": "state", "Domain":
"null", "Handle": "null", "Status": "null", "zip_code": "zip_code",
"address_1": "address_1", "address_2": "address_2", "auto_fuel": "null",
"auto_make": "null", "auto_size": "null", "auto_type": "null",
"auto_year": "null", "last_name": "last_name", "auto_based": "null",
"auto_class": "null", "auto_model": "null", "auto_style": "null",
"first_name": "first_name", "Syntax good": "null", "ahh_num_cars": "null",
"Invalid Reason": "null", "auto_make_model": "null", "confidence_code":
"null", "Catch All Domain": "null", "auto_manufacturer": "null",
"auto_purchase_date": "null", "auto_purchase_type": "null"}}'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27675#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/070.44f5e44199f261fb6b04f6e46267c55c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
