#26042: types for GenericForeignKey object_pk?
-------------------------------------+-------------------------------------
Reporter: brian | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by brian):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> Hello,
>
> The documentation suggests that object_id associated with
> GenericForeignKey should be a PositiveIntegerField. To be precise, the
> documentation says: "For most models, this means a PositiveIntegerField."
> but doesn't give any more details.
>
> However, this will not work for pk that are not integer. So for the case
> for django-guardian, where we want be more generic, we use a TextField.
>
> I notice that object_id in django.contrib.admin.models LogEntry is a
> TextField - although it doesn't use a GenericForeignKey.
>
> The problem with this is when making table joins on Postgresql -
> postgresql gets upset that the object_id is a string, and a different
> type on the destination table.
>
> e.g. simplified, we have a db model:
>
> {{{
> class UserObjectPermissionBase(models.Model):
> content_type = models.ForeignKey(ContentType)
> object_pk = models.CharField(_('object ID'), max_length=255)
> content_object = GenericForeignKey(fk_field='object_pk')
> }}}
>
> We lookup certain objects from this table. For simplicity, lets assume
> they *all* point to Group (django.contrib.auth) - i.e. content_type
> points to the Group content type - as this is where the test case fails.
> Also we use values, as we only are interested in the object_pk field.
>
> {{{
> user_obj_perms_queryset =
> UserObjectPermissionBase.objects.filter(....).values('object_pk')
> }}}
>
> Then we want to lookup all groups that the results of this queryset point
> to:
>
> {{{
> queryset.filter(Q(pk__in=user_obj_perms_queryset))
> }}}
>
> On sqlite and Mysql this works fine. On Postgresql it fails with a
> Postgresql error:
>
> {{{
> ======================================================================
> ERROR: test_empty_perms_sequence
> (guardian.testapp.tests.test_shortcuts.GetObjectsForUser)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/brian/tree/debian/upstream/django-
> guardian/guardian/testapp/tests/test_shortcuts.py", line 474, in
> test_empty_perms_sequence
> set(objects),
> File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
> 162, in __iter__
> self._fetch_all()
> File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
> 965, in _fetch_all
> self._result_cache = list(self.iterator())
> File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
> 238, in iterator
> results = compiler.execute_sql()
> File "/usr/lib/python2.7/dist-
> packages/django/db/models/sql/compiler.py", line 840, in execute_sql
> cursor.execute(sql, params)
> File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py",
> line 64, in execute
> return self.cursor.execute(sql, params)
> File "/usr/lib/python2.7/dist-packages/django/db/utils.py", line 97, in
> __exit__
> six.reraise(dj_exc_type, dj_exc_value, traceback)
> File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py",
> line 64, in execute
> return self.cursor.execute(sql, params)
> ProgrammingError: operator does not exist: integer = character varying
> LINE 1: ..."name" FROM "auth_group" WHERE ("auth_group"."id" IN
> (SELECT...
> ^
> HINT: No operator matches the given name and argument type(s). You might
> need to add explicit type casts.
> }}}
>
> The full SQL of the failing query in this case is:
>
> {{{
> SELECT "auth_group"."id", "auth_group"."name" FROM "auth_group" WHERE
> ("auth_group"."id" IN (SELECT U0."object_pk" FROM
> "guardian_userobjectpermission" U0 INNER JOIN "auth_permission" U2 ON (
> U0."permission_id" = U2."id" ) WHERE (U0."user_id" = 1 AND
> U2."content_type_id" = 2)) OR "auth_group"."id" IN (SELECT U0."object_pk"
> FROM "guardian_groupobjectpermission" U0 INNER JOIN "auth_group" U1 ON (
> U0."group_id" = U1."id" ) INNER JOIN "testapp_customuser_groups" U2 ON (
> U1."id" = U2."group_id" ) INNER JOIN "auth_permission" U4 ON (
> U0."permission_id" = U4."id" ) WHERE (U2."customuser_id" = 1 AND
> U4."content_type_id" = 2)))
> }}}
>
> Where I believe the problem is that "auth_group"."id" is integer and
> U0."object_pk" is a string, and they can't be compared.
>
> {{{
> ... WHERE ... "auth_group"."id" IN (SELECT U0."object_pk" FROM ...) ...
> }}}
>
> So is this a problem with our code? Maybe GenericForeignKey must have an
> integer value, which excludes it from non-integer pks? Or is this a
> problem with Django?
>
> Would be interested in feedback.
>
> Thanks
New description:
Hello,
The documentation suggests that object_id associated with
GenericForeignKey should be a PositiveIntegerField. To be precise, the
documentation says: "For most models, this means a PositiveIntegerField."
but doesn't give any more details.
However, this will not work for pk that are not integer. So for the case
for django-guardian, where we want be more generic, we use a TextField.
I notice that object_id in django.contrib.admin.models LogEntry is a
TextField - although it doesn't use a GenericForeignKey.
The problem with this is when making table joins on Postgresql -
postgresql gets upset that the object_id is a string, and a different type
on the destination table.
e.g. simplified, we have a db model:
{{{
class UserObjectPermissionBase(models.Model):
content_type = models.ForeignKey(ContentType)
object_pk = models.CharField(_('object ID'), max_length=255)
content_object = GenericForeignKey(fk_field='object_pk')
}}}
We lookup certain objects from this table. For simplicity, lets assume
they *all* point to Group (django.contrib.auth) - i.e. content_type
points to the Group content type - as this is where the test case fails.
Also we use values, as we only are interested in the object_pk field.
{{{
user_obj_perms_queryset =
UserObjectPermissionBase.objects.filter(....).values('object_pk')
}}}
Then we want to lookup all groups that the results of this queryset point
to:
{{{
queryset = Group.objects.all()
queryset.filter(Q(pk__in=user_obj_perms_queryset))
}}}
On sqlite and Mysql this works fine. On Postgresql it fails with a
Postgresql error:
{{{
======================================================================
ERROR: test_empty_perms_sequence
(guardian.testapp.tests.test_shortcuts.GetObjectsForUser)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/brian/tree/debian/upstream/django-
guardian/guardian/testapp/tests/test_shortcuts.py", line 474, in
test_empty_perms_sequence
set(objects),
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
162, in __iter__
self._fetch_all()
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
965, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line
238, in iterator
results = compiler.execute_sql()
File "/usr/lib/python2.7/dist-
packages/django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py",
line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python2.7/dist-packages/django/db/utils.py", line 97, in
__exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py",
line 64, in execute
return self.cursor.execute(sql, params)
ProgrammingError: operator does not exist: integer = character varying
LINE 1: ..."name" FROM "auth_group" WHERE ("auth_group"."id" IN (SELECT...
^
HINT: No operator matches the given name and argument type(s). You might
need to add explicit type casts.
}}}
The full SQL of the failing query in this case is:
{{{
SELECT "auth_group"."id", "auth_group"."name" FROM "auth_group" WHERE
("auth_group"."id" IN (SELECT U0."object_pk" FROM
"guardian_userobjectpermission" U0 INNER JOIN "auth_permission" U2 ON (
U0."permission_id" = U2."id" ) WHERE (U0."user_id" = 1 AND
U2."content_type_id" = 2)) OR "auth_group"."id" IN (SELECT U0."object_pk"
FROM "guardian_groupobjectpermission" U0 INNER JOIN "auth_group" U1 ON (
U0."group_id" = U1."id" ) INNER JOIN "testapp_customuser_groups" U2 ON (
U1."id" = U2."group_id" ) INNER JOIN "auth_permission" U4 ON (
U0."permission_id" = U4."id" ) WHERE (U2."customuser_id" = 1 AND
U4."content_type_id" = 2)))
}}}
Where I believe the problem is that "auth_group"."id" is integer and
U0."object_pk" is a string, and they can't be compared.
{{{
... WHERE ... "auth_group"."id" IN (SELECT U0."object_pk" FROM ...) ...
}}}
So is this a problem with our code? Maybe GenericForeignKey must have an
integer value, which excludes it from non-integer pks? Or is this a
problem with Django?
Would be interested in feedback.
Thanks
--
--
Ticket URL: <https://code.djangoproject.com/ticket/26042#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/063.0e78efa8728fec1a7cd9778cd11cf56b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
