1. I also could not find anything other than #26481 and a brief discussion <https://groups.google.com/d/msg/django-developers/xQCGx6MOZ8M/l42MypOCBAAJ> of it on the list.
2. I've often wished for something like this, but ended up resorting to assertNumQueries() for lack of a better solution. So yes, it'd certainly be useful to me/us, and I'd be curious to see how one might go about implementing it. 3. My gut says that an API, in Django core, that builds off of #26481 (e.g., .only(strict=True) and .defer(strict=True)) is the right direction (even for related fields), and my preference would be for a loud failure (i.e., an exception rather than a log message) when this "strict" mode is enabled. One can easily downgrade that to a log message when needed, less so the other way around. *Tobias McNulty*Chief Executive Officer [email protected] www.caktusgroup.com On Wed, Jan 3, 2018 at 12:33 PM, 'Bryan Helmig' via Django developers (Contributions to Django itself) <[email protected]> wrote: > At Zapier we're working with some rather complex and performance sensitive > QuerySet building (most currently around experimenting with GraphQL) and I > am constantly worried that the laziness of the ORM will surprise us in > production (after a few levels of related nesting, the math of a mistake > starts to really, really hurt). While not difficult to prevent, it does get > a bit tedious maintaining constant vigilance :tm:. I was curious if sealing > or locking a QuerySet has ever been seriously discussed before. > > For example, you might do something like: > > class Tag(models.Model): > name = models.CharField() > slug = models.SlugField() > # etc... > > def get_url(self): > return reverse('tag_index', args=[self.slug]) > > class Post(models.Model): > title = models.CharField() > tags = models.ManyToManyField(Tag) > # etc... > > posts = Post.objects.only( > 'id', 'title', > ).prefetch_related( > Prefetch('tags', queryset=Tag.objects.only('id', 'name')), > ) > > # we'd .seal() or .strict() and evaluate the queryset here: > for post in posts.seal(): > # get_url() would normally execute the classic N+1 queries > # but after qs.seal() would instead raise some exception > print ', '.join(tag.get_url() for tag in post.tags.all()) > > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > SealedQuerySetException: Cannot load sealed deferred attribute 'slug' on > prefetched Tag model. > > Of course the obvious solution is to just add 'slug' to only(), but in a > sufficiently complex application with many engineers working across various > app boundaries it is both difficult to predict and test against. It > requires lots of explicit self.assertNumQueries() and in the worse case can > cause "production surprise" as deeply nested QuerySets can potentially > explode in queries. > > This doesn't apply only to deferred attributes, but also related > OneToOne, ManyToOne, ManyToMany, etc. lazy resolution. FWIW, I would image > the FK/M2M N+1 pattern is a much more common surprise than opting into > .only() or .defer() as my example above outlines. > > I had a go at implementing it outside of Django core, but ended up having > to go down a monkey patching rabbit hole so I thought I'd reach out to the > mailing list. I'd imagine setting a flag when evaluating a QuerySet to > disable the laziness of a Model's fields and relations would be sufficient > (IE: if it isn't in cache, raise SealedQuerySetException). It also seems > backwards compatible, if you don't use .seal() you are still lazy like > before. > > So, I guess my questions are: > > 1. Are there any other past discussions around this topic I can look at? I > did find https://code.djangoproject.com/ticket/26481 which seems similar, > but doesn't mention relationships. > 2. Does this seem useful to anyone else? Specifically opting to raise > explicit exceptions instead of automatic (and sometimes surprising) queries. > 3. Anyone have tips on implementing this as a library external to Django > with lots of monkey patching? > > I'd be happy to take a swing at it if there was a >50% chance that it > would be at least considered. > > -bryan, cto & cofounder @ zapier > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/django-developers/962ca077-cde8-476e-92db- > 47adf5785866%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/962ca077-cde8-476e-92db-47adf5785866%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMGFDKROax3HfMEw1uQUX%2BD5hysHPb8ZrxmpJNK3PhV5bjem0A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
