[issue46121] Add a note to QueueListener documentation saying that SimpleQueue instances can't be used in multiprocessing scenarios

2021-12-18 Thread dmitry-bychkov 


New submission from dmitry-bychkov :

Currently, documentation for QueueListener 
(https://docs.python.org/3/library/logging.handlers.html#queuelistener) says:
"""
supports receiving logging messages from a queue, such as those implemented in 
the queue or multiprocessing modules.
"""

While also saying that we can use queue.SimpleQueue as an underlying queue:

"""
The queue is not required to have the task tracking API, which means that you 
can use queue.SimpleQueue instances for queue.
"""

I think it should also say that neither queue.SimpleQueue nor 
multiprocessing.SimpleQueue instances will work if your intended use of 
QueueListener is to receive logging messages from processes spawned by 
multiprocessing module. One should use multiprocessing.Queue instance.

usage of queue.SimpleQueue will result in pickle exception.
multiprocessing.SimpleQueue lacks the API QueueListener expects from queue.

English is not my first language so I'll step aside from PR's in documentation 
:)

--
assignee: docs@python
components: Documentation
messages: 408838
nosy: docs@python, iamdbychkov
priority: normal
severity: normal
status: open
title: Add a note to QueueListener documentation saying that SimpleQueue 
instances can't be used in multiprocessing scenarios
type: enhancement
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46121>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46163] multiprocessing logger deadlocks if used with logging.handlers.QueueHandler

2021-12-23 Thread dmitry-bychkov 


New submission from dmitry-bychkov :

Hello!
If you use multiprocessing logger with logging.handlers.QueueHandler it will 
deadlock on first attempt at logging.

First attempt will initialize queue's background thread, which uses 
multiprocessing logger itself which will result in something what looks like 
deadlock :) 

There are workarounds and I'm not sure if it can be classified as a bug, 
prehaps a little note in documentation about this behaviour will do? 

I've attached example script.

--
files: logger_deadlock.py
messages: 409088
nosy: iamdbychkov
priority: normal
severity: normal
status: open
title: multiprocessing logger deadlocks if used with 
logging.handlers.QueueHandler
versions: Python 3.9
Added file: https://bugs.python.org/file50513/logger_deadlock.py

___
Python tracker 
<https://bugs.python.org/issue46163>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com