#30114: Strange ValidationError issue
-------------------------------------+-------------------------------------
Reporter: Jerry | Owner: nobody
Vinokurov |
Type: Bug | Status: new
Component: Database | Version: 2.1
layer (models, ORM) | Keywords: validationerror,
Severity: Normal | uuid
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Our project uses UUIDs as primary keys on a number of models. This works
as expected for the majority of the time, but at some point, we run into
the following error:
{{{
ValidationError: ["'f0c9eb37-a073-47b5-bbb1-589920939c5e' is not a valid
UUID."]
}}}
This is in fact a valid UUID and also a UUID in our system. We use Sentry
for our error reporting, which gives the entire stack trace:
{{{
ValidationError: ["'670e5eaf-d231-305e-bb5b-8c8fb0262574' is not a valid
UUID."]
File "django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "python3.6/contextlib.py", line 52, in inner
return func(*args, **kwds)
File "django/contrib/admin/options.py", line 604, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
File "django/utils/decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "django/views/decorators/cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "django/contrib/admin/sites.py", line 223, in inner
return view(request, *args, **kwargs)
File "django/utils/decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "django/utils/decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "django/contrib/admin/options.py", line 1792, in changelist_view
'selection_note': _('0 of %(cnt)s selected') % {'cnt':
len(cl.result_list)},
File "django/db/models/query.py", line 250, in __len__
self._fetch_all()
File "django/db/models/query.py", line 1188, in _fetch_all
self._prefetch_related_objects()
File "django/db/models/query.py", line 723, in _prefetch_related_objects
prefetch_related_objects(self._result_cache,
*self._prefetch_related_lookups)
File "django/db/models/query.py", line 1569, in prefetch_related_objects
obj_list, additional_lookups = prefetch_one_level(obj_list,
prefetcher, lookup, level)
File "django/db/models/query.py", line 1682, in prefetch_one_level
prefetcher.get_prefetch_queryset(instances,
lookup.get_current_queryset(level)))
File "django/contrib/contenttypes/fields.py", line 194, in
get_prefetch_queryset
ret_val.extend(ct.get_all_objects_for_this_type(pk__in=fkeys))
File "django/db/models/query.py", line 268, in __iter__
self._fetch_all()
File "django/db/models/query.py", line 1186, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch,
chunk_size=self.chunk_size)
File "django/db/models/sql/compiler.py", line 1052, in execute_sql
sql, params = self.as_sql()
File "django/db/models/sql/compiler.py", line 464, in as_sql
where, w_params = self.compile(self.where) if self.where is not None
else ("", [])
File "django/db/models/sql/compiler.py", line 390, in compile
sql, params = node.as_sql(self, self.connection)
File "django/db/models/sql/where.py", line 81, in as_sql
sql, params = compiler.compile(child)
File "django/db/models/sql/compiler.py", line 390, in compile
sql, params = node.as_sql(self, self.connection)
File "django/db/models/lookups.py", line 355, in as_sql
return super().as_sql(compiler, connection)
File "django/db/models/lookups.py", line 163, in as_sql
rhs_sql, rhs_params = self.process_rhs(compiler, connection)
File "django/db/models/lookups.py", line 339, in process_rhs
sqls, sqls_params = self.batch_process_rhs(compiler, connection, rhs)
File "django/db/models/lookups.py", line 231, in batch_process_rhs
pre_processed = super().batch_process_rhs(compiler, connection, rhs)
File "django/db/models/lookups.py", line 51, in batch_process_rhs
_, params = self.get_db_prep_lookup(rhs, connection)
File "django/db/models/lookups.py", line 186, in get_db_prep_lookup
if self.get_db_prep_lookup_value_is_iterable else
File "django/db/models/lookups.py", line 185, in <listcomp>
[get_db_prep_value(v, connection, prepared=True) for v in value]
File "django/db/models/fields/__init__.py", line 2316, in
get_db_prep_value
value = self.to_python(value)
File "django/db/models/fields/__init__.py", line 2330, in to_python
params={'value': value},
}}}
Ok, so what's going on near line 2330 in
`django/db/models/fields/__init__.py` in the `to_python` function?
{{{
2325: return uuid.UUID(value)
2326: except (AttributeError, ValueError):
2327: raise exceptions.ValidationError(
2328: self.error_messages['invalid'],
2329: code='invalid',
2330: params={'value': value},
2331: )
2332: return value
2333: def formfield(self, **kwargs):
2334: return super().formfield(**{
}}}
And what's going on one frame before that, near line 2316 in
`django/db/models/fields/__init__.py` in `get_db_prep_value`:
{{{
2312: def get_db_prep_value(self, value, connection, prepared=False):
2313: if value is None:
2314: return None
2315: if not isinstance(value, uuid.UUID):
2316: value = self.to_python(value)
2317: if connection.features.has_native_uuid_field:
2318: return value
2319: return value.hex
}}}
Ok, so what's happening is that if the value that we're trying to convert
is not an instance of `uuid.UUID`, then we convert it to a Python value,
which calls `to_python`. This makes sense if `value` is a string, but...
Sentry is telling me that `value` ''is already'' a UUID:
`value UUID('f0c9eb37-a073-47b5-bbb1-589920939c5e')`
The check on line 2315 in `get_db_prep_value` should catch the fact that
this is a UUID, but it doesn't. I don't understand why at all. Sentry is
telling me that this is a UUID but it's failing the instance check; as a
result line 2325 in `to_python` attempts to convert the value to a UUID,
but `value` is already a UUID, so it throws a `ValidationError`.
That's the details of the error. The very weird thing about it is that
this error is totally inconsistent in when it shows up. We'll be running
for days in a production environment without any problems and then all of
a sudden this error will crop up. Once that happens, any attempt to
deserialize a UUID fails with this error.
We are running our application on an EC2 instance in AWS managed by
Elastic Beanstalk. A deployment of the application, which restarts it,
makes this error go away. But sooner or later it crops up again and brings
everything down. The worst thing is that I can't seem to replicate this
error in local development. Our DB runs on an RDS instance and I can point
my local development environment to that DB but I can't replicate the
error.
I'm truly baffled by this. Does anyone have any idea how this can happen?
It very much ''seems'' like a bug to me, hence this report, but I honestly
don't know. I'm just hoping to get some guidance or ideas about what is
going on here.
--
Ticket URL: <https://code.djangoproject.com/ticket/30114>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/054.6a36cfa5fcc6c4d84a21574360893b5a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
