Thanks Jason, I filed https://code.djangoproject.com/ticket/32111#ticket.
On Thursday, 15 October 2020 at 12:51:22 UTC+1 Jason wrote: > this is pretty interesting, and may be something to report to the devs at > https://groups.google.com/g/django-developers, or open up a ticket at > https://code.djangoproject.com/ I don't see any tickets with the query > *jsonfield > foreign data* that are similar with your experience, so you may have > found an edge case. > > On Thursday, October 15, 2020 at 5:31:09 AM UTC-4 shahee...@gmail.com > wrote: > >> >> Hi, >> >> I have a Django model working fine under Django 3.0 (i.e. "Django<3.1") >> which looks like this: >> >> =================== >> class Job(models.Model): >> id = models.CharField(max_length=36, primary_key=True) >> queue = models.CharField(max_length=40) >> args = JSONField() >> kwargs = JSONField() >> type = models.CharField(max_length=80) >> ... >> >> class Meta: >> managed = False # <------ The table is implemented as a >> Postgres FDW wrapper. >> db_table = 'jobs' >> =================== >> >> I am testing the update to Django 3.1.2 and hit an error in executing >> this line: >> >> jobs = list(models.Job.objects.filter(queue='celery', >> state='scheduled')) >> >> The error is as follows from pytest (i.e. stack trace with local >> variables too): >> >> ================== >> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ >> _ _ _ >> /usr/local/lib/python3.8/dist-packages/django/db/models/query.py:287: in >> __iter__ >> self._fetch_all() >> self = <QuerySet []> >> /usr/local/lib/python3.8/dist-packages/django/db/models/query.py:1308: in >> _fetch_all >> self._result_cache = list(self._iterable_class(self)) >> self = <QuerySet []> >> /usr/local/lib/python3.8/dist-packages/django/db/models/query.py:70: in >> __iter__ >> for row in compiler.results_iter(results): >> annotation_col_map = {} >> compiler = <django.db.models.sql.compiler.SQLCompiler object at >> 0x7f8685e49160> >> db = 'fdw' >> init_list = ['id', 'queue', 'args', 'kwargs', 'type', 'state', >> ...] >> klass_info = {'model': <class 'paiyroll.models.batch.Job'>, >> 'select_fields': [0, 1, 2, 3, 4, 5, ...]} >> known_related_objects = [] >> model_cls = <class 'paiyroll.models.batch.Job'> >> model_fields_end = 9 >> model_fields_start = 0 >> queryset = <QuerySet []> >> related_populators = [] >> results = [[('8f4ab6b0-914f-4a75-972d-febbe55011fc', 'celery', >> ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], {}, >> 'paiyroll.tasks.function_run', 'scheduled', ...)]] >> select = [(Col(jobs, paiyroll.Job.id), ('"jobs"."id"', []), >> None), (Col(jobs, paiyroll.Job.queue), ('"jobs"."queue"', []), None..., >> paiyroll.Job.type), ('"jobs"."type"', []), None), (Col(jobs, >> paiyroll.Job.state), ('"jobs"."state"', []), None), ...] >> select_fields = [0, 1, 2, 3, 4, 5, ...] >> self = <django.db.models.query.ModelIterable object at >> 0x7f86836f3040> >> /usr/local/lib/python3.8/dist-packages/django/db/models/sql/compiler.py:1100: >> >> in apply_converters >> value = converter(value, expression, connection) >> connection = <django.db.backends.postgresql.base.DatabaseWrapper >> object at 0x7f869a321670> >> converter = <bound method JSONField.from_db_value of >> <django.contrib.postgres.fields.jsonb.JSONField: args>> >> converters = [(2, ([<bound method JSONField.from_db_value of >> <django.contrib.postgres.fields.jsonb.JSONField: args>>], Col(jobs, >> pa...NField.from_db_value of >> <django.contrib.postgres.fields.jsonb.JSONField: details>>], Col(jobs, >> paiyroll.Job.details)))] >> convs = [<bound method JSONField.from_db_value of >> <django.contrib.postgres.fields.jsonb.JSONField: args>>] >> expression = Col(jobs, paiyroll.Job.args) >> pos = 2 >> row = ['8f4ab6b0-914f-4a75-972d-febbe55011fc', 'celery', >> ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], {}, >> 'paiyroll.tasks.function_run', 'scheduled', ...] >> rows = <itertools.chain object at 0x7f8683ae7520> >> self = <django.db.models.sql.compiler.SQLCompiler object at >> 0x7f8685e49160> >> value = ['paiyroll.tasks', 'call_to_string', >> 'call_to_string', [], {}] >> >> >> >> >> >> */usr/local/lib/python3.8/dist-packages/django/db/models/fields/json.py:74: >> in from_db_value return json.loads(value, cls=self.decoder) >> connection = <django.db.backends.postgresql.base.DatabaseWrapper >> object at 0x7f869a321670> expression = Col(jobs, paiyroll.Job.args) >> self = <django.contrib.postgres.fields.jsonb.JSONField: args> >> value = ['paiyroll.tasks', 'call_to_string', 'call_to_string', >> [], {}] * >> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ >> _ _ _ >> >> s = ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], cls = >> None >> object_hook = None, parse_float = None, parse_int = None, parse_constant >> = None >> object_pairs_hook = None, kw = {} >> >> def loads(s, *, cls=None, object_hook=None, parse_float=None, >> parse_int=None, parse_constant=None, object_pairs_hook=None, >> **kw): >> """Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` >> instance >> containing a JSON document) to a Python object. >> >> ``object_hook`` is an optional function that will be called with >> the >> result of any object literal decode (a ``dict``). The return value >> of >> ``object_hook`` will be used instead of the ``dict``. This feature >> can be used to implement custom decoders (e.g. JSON-RPC class >> hinting). >> >> ``object_pairs_hook`` is an optional function that will be called >> with the >> result of any object literal decoded with an ordered list of >> pairs. The >> return value of ``object_pairs_hook`` will be used instead of the >> ``dict``. >> This feature can be used to implement custom decoders. If >> ``object_hook`` >> is also defined, the ``object_pairs_hook`` takes priority. >> >> ``parse_float``, if specified, will be called with the string >> of every JSON float to be decoded. By default this is equivalent >> to >> float(num_str). This can be used to use another datatype or parser >> for JSON floats (e.g. decimal.Decimal). >> >> ``parse_int``, if specified, will be called with the string >> of every JSON int to be decoded. By default this is equivalent to >> int(num_str). This can be used to use another datatype or parser >> for JSON integers (e.g. float). >> >> ``parse_constant``, if specified, will be called with one of the >> following strings: -Infinity, Infinity, NaN. >> This can be used to raise an exception if invalid JSON numbers >> are encountered. >> >> To use a custom ``JSONDecoder`` subclass, specify it with the >> ``cls`` >> kwarg; otherwise ``JSONDecoder`` is used. >> >> The ``encoding`` argument is ignored and deprecated since Python >> 3.1. >> """ >> if isinstance(s, str): >> if s.startswith('\ufeff'): >> raise JSONDecodeError("Unexpected UTF-8 BOM (decode using >> utf-8-sig)", >> s, 0) >> else: >> if not isinstance(s, (bytes, bytearray)): >> > raise TypeError(f'the JSON object must be str, bytes or >> bytearray, ' >> f'not {s.__class__.__name__}') >> E TypeError: the JSON object must be str, bytes or >> bytearray, not list >> >> cls = None >> kw = {} >> object_hook = None >> object_pairs_hook = None >> parse_constant = None >> parse_float = None >> parse_int = None >> s = ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], >> {}] >> =============================== >> >> As you can perhaps see from the bolded part (for >> /usr/local/lib/python3.8/dist-packages/django/db/models/fields/json.py:74), >> the value being written into the JSONField called "args" is a Python list, >> (shown as "s" on the last line of the traceback-with-values). I am aware >> of documented changes around serializers but did not think that affected >> me; however, given that I am using an FDW to return the data, is it that I >> should now be serialising returned data into a string? >> >> Any pointers appreciated. >> >> Thanks, Shaheed >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f1392121-70f6-4359-8d2c-10dbb157607bn%40googlegroups.com.
