New submission from Freek Dijkstra <fr...@macfreek.nl>:
Summary:
multiprocessing.Pool contains a bug when the script is invoked with pdb.
Steps to reproduce:
Consider the following script:
```
from multiprocessing import Pool
def f(x):
return x*x
if __name__ == '__main__':
with Pool(5) as p:
print(p.map(f, [1, 2, 3]))
```
When called as `python3.10 test.py`, this works fine, and returns `[1, 4, 9]`
When called as `python3.10 -m pdb test.py`, it gives the following exception:
```
freek@minnie ~ ยป python3.9 -m pdb test.py
> /Users/freek/test.py(3)<module>()
-> from multiprocessing import Pool
(Pdb) c
Traceback (most recent call last):
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py",
line 1704, in main
pdb._runscript(mainpyfile)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pdb.py",
line 1573, in _runscript
self.run(statement)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/bdb.py",
line 580, in run
exec(cmd, globals, locals)
File "<string>", line 1, in <module>
File "/Users/freek/test.py", line 3, in <module>
from multiprocessing import Pool
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py",
line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py",
line 212, in __init__
self._repopulate_pool()
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py",
line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py",
line 326, in _repopulate_pool_static
w.start()
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py",
line 121, in start
self._popen = self._Popen(self)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py",
line 284, in _Popen
return Popen(process_obj)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py",
line 32, in __init__
super().__init__(process_obj)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py",
line 19, in __init__
self._launch(process_obj)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py",
line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py",
line 183, in get_preparation_data
main_mod_name = getattr(main_module.__spec__, "name", None)
AttributeError: module '__main__' has no attribute '__spec__'
```
I can reproduce this in Python 3.8.7, 3.9.1, 3.10.0a3. It works fine in Python
3.7.9.
A workaround is to define `__spec__` in __main__:
```
from multiprocessing import Pool
def f(x):
return x*x
if __name__ == '__main__':
__spec__ = None
with Pool(5) as p:
print(p.map(f, [1, 2, 3]))
```
I'd say that multiprocessing/spawn.py", line 183, in get_preparation_data is
flawed
main_mod_name = getattr(main_module.__spec__, "name", None)
----------
components: Library (Lib)
messages: 385168
nosy: macfreek
priority: normal
severity: normal
status: open
title: pdb & multiprocessing.Pool: AttributeError: module '__main__' has no
attribute '__spec__'
versions: Python 3.10, Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue42949>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
