#24042: Custom AutoField fields do not work correctly on postgres
-------------------------------+--------------------------------------
Reporter: lucaswiman | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by lucaswiman):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> The documentation (https://docs.djangoproject.com/en/dev/howto/custom-
> model-fields/#emulating-built-in-field-types) on Field.get_internal_type
> implies that it should define custom behavior and ensure that database-
> correct behavior is followed. For Postgres (though not for sqlite),
> specifying get_internal_type() to be "AutoField" fails to correctly set
> the field, leading to errors like this:
>
> {{{
> test_auto_field (field_subclassing.tests.CustomField) ... ERROR
>
> ======================================================================
> ERROR: test_auto_field (field_subclassing.tests.CustomField)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/Users/lucaswiman/opensource/django/tests/field_subclassing/tests.py",
> line 128, in test_auto_field
> o.save()
> File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
> line 694, in save
> force_update=force_update, update_fields=update_fields)
> File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
> line 722, in save_base
> updated = self._save_table(raw, cls, force_insert, force_update,
> using, update_fields)
> File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
> line 803, in _save_table
> result = self._do_insert(cls._base_manager, using, fields, update_pk,
> raw)
> File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
> line 842, in _do_insert
> using=using, raw=raw)
> File "/Users/lucaswiman/opensource/django/django/db/models/manager.py",
> line 131, in manager_method
> return getattr(self.get_queryset(), name)(*args, **kwargs)
> File "/Users/lucaswiman/opensource/django/django/db/models/query.py",
> line 954, in _insert
> return query.get_compiler(using=using).execute_sql(return_id)
> File
> "/Users/lucaswiman/opensource/django/django/db/models/sql/compiler.py",
> line 931, in execute_sql
> cursor.execute(sql, params)
> File "/Users/lucaswiman/opensource/django/django/db/backends/utils.py",
> line 65, in execute
> return self.cursor.execute(sql, params)
> File "/Users/lucaswiman/opensource/django/django/db/utils.py", line 95,
> in __exit__
> six.reraise(dj_exc_type, dj_exc_value, traceback)
> File "/Users/lucaswiman/opensource/django/django/db/backends/utils.py",
> line 65, in execute
> return self.cursor.execute(sql, params)
> IntegrityError: null value in column "id" violates not-null constraint
> DETAIL: Failing row contains (null).
> }}}
>
> The attached diff adds a regression test which fails on postgres
> databases with the error above. I also fixed all other uses of
> isinstance(..., AutoField) I could find in the code.
>
> The immediate motivation for this is writing integration tests which use
> django-salesforce on a CI system that uses a postgres database. I've
> filed a separate issue there, though I think fixing the behavior in
> Django to match the documentation would be a preferable solution. The
> package maintainer for django-salesforce agrees (https://github.com
> /django-salesforce/django-salesforce/issues/90#issuecomment-67932443).
>
> '''How to Reproduce'''
>
> Check out the master branch and apply my diff from the tests/ directory.
> Run the tests with a settings file pointing to a postgres database:
> {{{
> PYTHONPATH=..:$PYTHONPATH ./runtests.py --verbosity=2 --failfast
> field_subclassing --settings=test_postgres
> }}}
> It will fail with the above stack trace.
>
> '''Testing'''
>
> I verified that the regression test fails without the remainder of the
> diff. I also verified that the tests still pass on the default
> test_sqlite file.
New description:
Patch also available at https://github.com/django/django/pull/3776
The documentation (https://docs.djangoproject.com/en/dev/howto/custom-
model-fields/#emulating-built-in-field-types) on Field.get_internal_type
implies that it should define custom behavior and ensure that database-
correct behavior is followed. For Postgres (though not for sqlite),
specifying get_internal_type() to be "AutoField" fails to correctly set
the field, leading to errors like this:
{{{
test_auto_field (field_subclassing.tests.CustomField) ... ERROR
======================================================================
ERROR: test_auto_field (field_subclassing.tests.CustomField)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/lucaswiman/opensource/django/tests/field_subclassing/tests.py",
line 128, in test_auto_field
o.save()
File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
line 694, in save
force_update=force_update, update_fields=update_fields)
File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
line 722, in save_base
updated = self._save_table(raw, cls, force_insert, force_update,
using, update_fields)
File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
line 803, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk,
raw)
File "/Users/lucaswiman/opensource/django/django/db/models/base.py",
line 842, in _do_insert
using=using, raw=raw)
File "/Users/lucaswiman/opensource/django/django/db/models/manager.py",
line 131, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/Users/lucaswiman/opensource/django/django/db/models/query.py",
line 954, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File
"/Users/lucaswiman/opensource/django/django/db/models/sql/compiler.py",
line 931, in execute_sql
cursor.execute(sql, params)
File "/Users/lucaswiman/opensource/django/django/db/backends/utils.py",
line 65, in execute
return self.cursor.execute(sql, params)
File "/Users/lucaswiman/opensource/django/django/db/utils.py", line 95,
in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Users/lucaswiman/opensource/django/django/db/backends/utils.py",
line 65, in execute
return self.cursor.execute(sql, params)
IntegrityError: null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null).
}}}
The attached diff adds a regression test which fails on postgres databases
with the error above. I also fixed all other uses of isinstance(...,
AutoField) I could find in the code.
The immediate motivation for this is writing integration tests which use
django-salesforce on a CI system that uses a postgres database. I've filed
a separate issue there, though I think fixing the behavior in Django to
match the documentation would be a preferable solution. The package
maintainer for django-salesforce agrees (https://github.com/django-
salesforce/django-salesforce/issues/90#issuecomment-67932443).
'''How to Reproduce'''
Check out the master branch and apply my diff from the tests/ directory.
Run the tests with a settings file pointing to a postgres database:
{{{
PYTHONPATH=..:$PYTHONPATH ./runtests.py --verbosity=2 --failfast
field_subclassing --settings=test_postgres
}}}
It will fail with the above stack trace.
'''Testing'''
I verified that the regression test fails without the remainder of the
diff. I also verified that the tests still pass on the default test_sqlite
file.
--
Comment:
Pat
--
Ticket URL: <https://code.djangoproject.com/ticket/24042#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/068.a30fc71cf82d9e9e30a645605c8cca69%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
