You really should use an async task worker like Celery for this, to get the scraping outside of Django's request-response loop.
Is the urlopen happening with any specific URL or seemingly at random? -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/5fe18bd7-4501-4ac9-9a44-fa536dafdd8b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
