On 8/27/2017 11:54 AM, Peter Otten wrote:
The documentation
https://www.crummy.com/software/BeautifulSoup/bs4/doc/#making-the-soup
says you can make the BeautifulSoup object from a string or file.
Can you give a few more details where the queue comes into play? A small
code sample would be ideal.
A worker thread uses a request object to get the page and puts it into
queue as page.content (HTML). Another worker thread gets the
page.content from the queue to apply BeautifulSoup and nothing happens.
soup = BeautifulSoup(page_content, 'lxml')
print(soup)
No output whatsoever. If I remove 'lxml', I get the UserWarning that no
parser wasn't explicitly set and get the reference to threading.py at
line 80.
I verified that page.content that goes into and out of the queue is the
same page.content that goes into and out of a list.
I read somewhere that BeautifulSoup may not be thread-safe. I've never
had a problem with threads storing the output into a queue. Using a
queue (random order) instead of a list (sequential order) to feed pages
for the input is making it wonky.
Chris R.
--
https://mail.python.org/mailman/listinfo/python-list
