On Saturday, 12 November 2011 23:04:44 UTC, Emil Sandin wrote:
>
> Hi, I am running a Django app on appengine, but I am constantly
> hitting the roof with read operations.
> I have not found an easy way to find what's causing this, so I am
> trying to find it out myself.
> To me, it seems that even if I use all().filter(..) or get(), the
> database is queried for all instances.
>
> I wrote a custom manager like this:
>
> class LoggingManager(models.Manager):
> def get_query_set(self):
> query_set = super(SignalsManager, self).get_query_set()
> logging.debug("Got the following queryset from the backend:")
> logging.debug(query_set)
> if query_set.count():
> key = "num_of_reads_for_" + str(type(query_set[0]))
> num_of_calls_so_far = memcache.get(key) or 0
> num_of_calls_so_far += query_set.count()
> memcache.delete(key)
> memcache.add(key, num_of_calls_so_far )
> logging.debug("++++++++++++++++++++++++++++++++++")
> logging.debug("total number of reads for " + key)
> logging.debug(num_of_calls_so_far)
> logging.debug("----------------------------------")
> return query_set
>
> I then created a class:
>
> class Foo(models.Model):
> bar = models.CharField(max_length=10)
> objects = LoggingManager()
>
> When I tried this in the shell I got:
>
> >>> Foo(bar="Hi").save()
> >>> Foo(bar="there").save()
> >>> Foo.objects.get(bar="Hi")
> DEBUG:root:Got the following queryset from the backend:
> DEBUG:root:[<Foo: Foo object>, <Foo: Foo object>]
> DEBUG:root:++++++++++++++++++++++++++++++++++
> DEBUG:root:total number of reads for num_of_reads_for_<class
> 'models.Foo'>
> DEBUG:root:2
> DEBUG:root:----------------------------------
> <Foo: Foo object>
> >>> Foo.objects.all().filter(bar="Hi")
> DEBUG:root:Got the following queryset from the backend:
> DEBUG:root:[<Foo: Foo object>, <Foo: Foo object>]
> DEBUG:root:++++++++++++++++++++++++++++++++++
> DEBUG:root:total number of reads for num_of_reads_for_<class
> 'models.Foo'>
> DEBUG:root:4
> DEBUG:root:----------------------------------
> [<Foo: Foo object>]
> >>> Foo.objects.all().filter(bar="NotExisting")
> DEBUG:root:Got the following queryset from the backend:
> DEBUG:root:[<Foo: Foo object>, <Foo: Foo object>]
> DEBUG:root:++++++++++++++++++++++++++++++++++
> DEBUG:root:total number of reads for num_of_reads_for_<class
> 'models.Foo'>
> DEBUG:root:6
> DEBUG:root:----------------------------------
> []
>
> According to the Django documentation, 'the act of creating a QuerySet
> doesn't involve any database activity'.
> But since the queryset contains two instances of Foo object when I use
> the get() method or all().filter(), isn't the database queried for all
> Foo instances? And then the queryset is filtered?
> Am I doing something wrong here? Is there a better way to do this?
> And does anybody know how this would affect App Engines Quote Details,
> since they have a limit of number of operations?
>
> I am using:
> Python 2.5.1,
> Django (1, 3, 0, 'alpha', 1)
This message is probably off-topic for this list, as Django running on
AppEngine is not really the same as Django standalone - it depends on
django-nonrel to talk to the nonrelational datastore, so database behaviour
is very different. You would probably be better off asking on a specific
django-nonrel mailing list.
In ordinary Django, each call to LoggingManager would result in a single
database hit, which would be executed when it reaches the first logging
call, since in order to output the string representation of the QuerySet
Django would need to actually evaluate it. The call to type(query_set[0])
would not cause another hit, because Django has already stored the values
inside the QuerySet's internal cache.
Clearly django-nonrel is doing something different here, but I don't have
enough experience with it to tell you what. I can say that in plain
AppEngine, without Django, you can call `queryset.kind()` to get the
queryset's model without any database access, but I don't know if that will
work with django-nonrel.
--
DR.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/django-users/-/ybXDsP_RP1EJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
