Yes! i can load the Url
("http://www.aaasports.co.kr/front/productlist.php?code=0010&brandcode=2&listnum=30&sort=&block=0&gotopage=1";)
without issue. I think the time to load the URL is longer than any other
URL, but it does not take much long.
I searched for the Postman you suggested
Are you able to load the URL directly in the browser without issue? How
about using an API client like Postman?
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
Thanks.!
I had better use an async task worker! i need to study the worker...
the error happens with a specific URL.
2018년 1월 9일 화요일 오후 8시 7분 15초 UTC+9, Jason 님의 말:
>
> You really should use an async task worker like Celery for this, to get
> the scraping outside of Django's request-response
You really should use an async task worker like Celery for this, to get the
scraping outside of Django's request-response loop.
Is the urlopen happening with any specific URL or seemingly at random?
--
You received this message because you are subscribed to the Google Groups
"Django users" gro
yes, i use a urlopen to crawl the some site.
like this:
html = urlopen(page)
bs0bj = BeautifulSoup(html, "html.parser")
and error happen in " /usr/lib/python3.5/urllib/request.py in do_open, "
when i open 'the site', the loading time is longer than others. then others
which i crawl dont
Hello,
Where does this happen? Do you do urlopen somewhere? (IIRC, changing Django
settings doesn't affect internal Python objects).
urllib.request has a timeout, it's the second parameter after the data:
https://docs.python.org/3/library/urllib.request.html?highlight=urllib#urllib.request.urlope
6 matches
Mail list logo