New submission from Nikita Pchelin <nikita.pche...@gmail.com>:
I've wrote a little application that uses multiprocessing module:
https://github.com/jango/PC/blob/master/pc/pc-example.py
When I run it in my Linux setup, I get the expected output (Python 2.7.1+):
2011-12-09 14:16:29,014 Started Producer 0.
2011-12-09 14:16:29,076 Started Producer 1.
2011-12-09 14:16:29,076 Producer process (0) exited...
2011-12-09 14:16:29,131 Started Producer 2.
2011-12-09 14:16:29,188 Started Producer 3.
2011-12-09 14:16:29,218 Started Producer 4.
2011-12-09 14:16:29,281 Started Producer 5.
2011-12-09 14:16:29,327 Started Producer 6.
2011-12-09 14:16:29,377 Started Producer 7.
2011-12-09 14:16:29,403 Producer process (1) exited...
2011-12-09 14:16:29,412 Started Producer 8.
2011-12-09 14:16:29,504 Started Producer 9.
2011-12-09 14:16:29,528 Producer process (4) exited...
2011-12-09 14:16:29,570 Started Producer 10.
2011-12-09 14:16:29,615 Started Producer 11.
2011-12-09 14:16:29,622 Producer process (3) exited...
2011-12-09 14:16:29,637 Producer process (7) exited...
2011-12-09 14:16:29,653 Producer process (5) exited...
2011-12-09 14:16:29,692 Started Producer 12.
2011-12-09 14:16:29,731 Producer process (12) exited...
2011-12-09 14:16:29,747 Started Producer 13.
2011-12-09 14:16:29,809 Started Producer 14.
2011-12-09 14:16:29,860 Started Producer 15.
2011-12-09 14:16:29,903 Producer process (2) exited...
2011-12-09 14:16:29,905 Started Producer 16.
2011-12-09 14:16:29,918 Producer process (8) exited...
2011-12-09 14:16:29,970 Started Producer 17.
2011-12-09 14:16:30,001 Started Producer 18.
2011-12-09 14:16:30,070 Started Producer 19.
2011-12-09 14:16:30,090 Producer process (11) exited...
2011-12-09 14:16:30,105 Producer process (10) exited...
2011-12-09 14:16:30,137 Started Producer 20.
2011-12-09 14:16:30,152 Producer process (6) exited...
2011-12-09 14:16:30,183 Producer process (9) exited...
2011-12-09 14:16:30,183 Producer process (17) exited...
2011-12-09 14:16:30,251 Started Producer 21.
2011-12-09 14:16:30,288 Started Producer 22.
2011-12-09 14:16:30,308 Producer process (18) exited...
2011-12-09 14:16:30,308 Producer process (20) exited...
2011-12-09 14:16:30,355 Producer process (16) exited...
2011-12-09 14:16:30,380 Started Producer 23.
2011-12-09 14:16:30,436 Started Producer 24.
2011-12-09 14:16:30,472 Started Producer 25.
2011-12-09 14:16:30,480 Producer process (22) exited...
2011-12-09 14:16:30,509 Started Producer 26.
2011-12-09 14:16:30,554 Started Producer 27.
2011-12-09 14:16:30,609 Started Producer 28.
2011-12-09 14:16:30,620 Producer process (14) exited...
2011-12-09 14:16:30,636 Producer process (15) exited...
2011-12-09 14:16:30,660 Started Producer 29.
2011-12-09 14:16:30,667 Producer process (19) exited...
2011-12-09 14:16:30,667 Producer process (13) exited...
2011-12-09 14:16:30,667 Producer process (27) exited...
2011-12-09 14:16:30,750 Started Consumer 0.
2011-12-09 14:16:30,761 --> 0 produces 0.
2011-12-09 14:16:30,776 Producer process (28) exited...
2011-12-09 14:16:30,770 --> 1 produces 10.
2011-12-09 14:16:30,783 --> 4 produces 40.
2011-12-09 14:16:30,797 Started Consumer 1.
2011-12-09 14:16:30,807 --> 3 produces 30.
2011-12-09 14:16:30,816 --> 5 produces 50.
2011-12-09 14:16:30,817 --> 12 produces 120.
2011-12-09 14:16:30,819 --> 2 produces 20.
2011-12-09 14:16:30,826 --> 8 produces 80.
2011-12-09 14:16:30,835 --> 11 produces 110.
2011-12-09 14:16:30,848 --> 10 produces 100.
2011-12-09 14:16:30,849 --> 6 produces 60.
2011-12-09 14:16:30,852 --> 17 produces 170.
2011-12-09 14:16:30,853 --> 9 produces 90.
2011-12-09 14:16:30,859 --> 18 produces 180.
2011-12-09 14:16:30,860 --> 20 produces 200.
2011-12-09 14:16:30,865 --> 16 produces 160.
2011-12-09 14:16:30,866 --> 22 produces 220.
2011-12-09 14:16:30,867 --> 14 produces 140.
2011-12-09 14:16:30,868 --> 15 produces 150.
2011-12-09 14:16:30,869 --> 19 produces 190.
2011-12-09 14:16:30,874 --> 13 produces 130.
2011-12-09 14:16:30,823 Producer process (21) exited...
2011-12-09 14:16:30,812 --> 7 produces 70.
2011-12-09 14:16:30,893 --> 28 produces 280.
2011-12-09 14:16:30,854 Producer process (25) exited...
2011-12-09 14:16:30,894 Started Consumer 2.
2011-12-09 14:16:30,875 --> 27 produces 270.
2011-12-09 14:16:30,905 --> 25 produces 250.
2011-12-09 14:16:30,917 --> 21 produces 210.
2011-12-09 14:16:30,943 Started Consumer 3.
2011-12-09 14:16:31,002 Started Consumer 4.
2011-12-09 14:16:31,049 Started Consumer 5.
2011-12-09 14:16:31,091 Started Consumer 6.
2011-12-09 14:16:31,137 Started Consumer 7.
2011-12-09 14:16:31,197 Producer process (23) exited...
2011-12-09 14:16:31,213 --> 23 produces 230.
2011-12-09 14:16:31,206 Started Consumer 8.
2011-12-09 14:16:31,263 Started Consumer 9.
2011-12-09 14:16:31,275 Producer process (24) exited...
2011-12-09 14:16:31,307 Started Consumer 10.
2011-12-09 14:16:31,322 --> 24 produces 240.
2011-12-09 14:16:31,357 Started Consumer 11.
2011-12-09 14:16:31,386 Started Consumer 12.
2011-12-09 14:16:31,400 Producer process (26) exited...
2011-12-09 14:16:31,432 Started Consumer 13.
2011-12-09 14:16:31,431 --> 26 produces 260.
2011-12-09 14:16:31,476 Started Consumer 14.
2011-12-09 14:16:31,527 Started Consumer 15.
2011-12-09 14:16:31,556 Producer process (29) exited...
2011-12-09 14:16:31,570 Started Consumer 16.
2011-12-09 14:16:31,587 --> 29 produces 290.
2011-12-09 14:16:31,630 Started Consumer 17.
2011-12-09 14:16:31,664 Started Consumer 18.
2011-12-09 14:16:31,714 Started Consumer 19.
2011-12-09 14:16:31,768 Started Consumer 20.
2011-12-09 14:16:31,815 Started Consumer 21.
2011-12-09 14:16:31,847 Started Consumer 22.
2011-12-09 14:16:31,903 Started Consumer 23.
2011-12-09 14:16:31,932 Started Consumer 24.
2011-12-09 14:16:31,981 Started Consumer 25.
2011-12-09 14:16:32,023 Started Consumer 26.
2011-12-09 14:16:32,073 Started Consumer 27.
2011-12-09 14:16:32,121 Started Consumer 28.
2011-12-09 14:16:32,175 Started Consumer 29.
2011-12-09 14:16:32,181 Consumer (0): instance received shutdown call...
2011-12-09 14:16:32,182 Consumer (1): instance received shutdown call...
2011-12-09 14:16:32,185 Consumer (2): instance received shutdown call...
2011-12-09 14:16:32,186 Consumer (3): instance received shutdown call...
2011-12-09 14:16:32,187 Consumer (4): instance received shutdown call...
2011-12-09 14:16:32,188 Consumer (5): instance received shutdown call...
2011-12-09 14:16:32,189 Consumer (6): instance received shutdown call...
2011-12-09 14:16:32,190 Consumer (7): instance received shutdown call...
2011-12-09 14:16:32,191 Consumer (8): instance received shutdown call...
2011-12-09 14:16:32,192 Consumer (9): instance received shutdown call...
2011-12-09 14:16:32,193 Consumer (10): instance received shutdown call...
2011-12-09 14:16:32,194 Consumer (11): instance received shutdown call...
2011-12-09 14:16:32,195 Consumer (12): instance received shutdown call...
2011-12-09 14:16:32,196 Consumer (13): instance received shutdown call...
2011-12-09 14:16:32,197 Consumer (14): instance received shutdown call...
2011-12-09 14:16:32,198 Consumer (15): instance received shutdown call...
2011-12-09 14:16:32,199 Consumer (16): instance received shutdown call...
2011-12-09 14:16:32,201 Consumer (17): instance received shutdown call...
2011-12-09 14:16:32,207 Consumer (18): instance received shutdown call...
2011-12-09 14:16:32,208 Consumer (19): instance received shutdown call...
2011-12-09 14:16:32,209 Consumer (20): instance received shutdown call...
2011-12-09 14:16:32,210 Consumer (21): instance received shutdown call...
2011-12-09 14:16:32,211 Consumer (22): instance received shutdown call...
2011-12-09 14:16:32,212 Consumer (23): instance received shutdown call...
2011-12-09 14:16:32,213 Consumer (24): instance received shutdown call...
2011-12-09 14:16:32,213 Consumer (25): instance received shutdown call...
2011-12-09 14:16:32,214 Consumer (26): instance received shutdown call...
2011-12-09 14:16:32,215 Consumer (27): instance received shutdown call...
2011-12-09 14:16:32,216 Consumer (28): instance received shutdown call...
2011-12-09 14:16:32,217 Consumer (29): instance received shutdown call...
2011-12-09 14:16:32,226 Last call to consumer.
2011-12-09 14:16:32,231 Consumer instance (8) exited...
2011-12-09 14:16:32,274 Last call to consumer.
2011-12-09 14:16:32,276 Consumer instance (9) exited...
2011-12-09 14:16:32,337 Last call to consumer.
2011-12-09 14:16:32,338 Consumer instance (10) exited...
2011-12-09 14:16:32,367 Last call to consumer.
2011-12-09 14:16:32,371 Consumer instance (11) exited...
2011-12-09 14:16:32,399 Last call to consumer.
2011-12-09 14:16:32,404 Consumer instance (12) exited...
2011-12-09 14:16:32,440 Last call to consumer.
2011-12-09 14:16:32,441 Consumer instance (13) exited...
2011-12-09 14:16:32,492 Last call to consumer.
2011-12-09 14:16:32,504 Consumer instance (14) exited...
2011-12-09 14:16:32,539 Last call to consumer.
2011-12-09 14:16:32,540 Consumer instance (15) exited...
2011-12-09 14:16:32,598 Last call to consumer.
2011-12-09 14:16:32,599 Consumer instance (16) exited...
2011-12-09 14:16:32,633 Last call to consumer.
2011-12-09 14:16:32,634 Consumer instance (17) exited...
2011-12-09 14:16:32,679 Last call to consumer.
2011-12-09 14:16:32,687 Consumer instance (18) exited...
2011-12-09 14:16:32,726 Last call to consumer.
2011-12-09 14:16:32,727 Consumer instance (19) exited...
2011-12-09 14:16:32,789 Last call to consumer.
2011-12-09 14:16:32,790 Consumer instance (20) exited...
2011-12-09 14:16:32,820 Last call to consumer.
2011-12-09 14:16:32,833 Consumer instance (21) exited...
2011-12-09 14:16:32,867 Last call to consumer.
2011-12-09 14:16:32,873 Consumer instance (22) exited...
2011-12-09 14:16:32,913 Last call to consumer.
2011-12-09 14:16:32,918 Consumer instance (23) exited...
2011-12-09 14:16:32,936 Last call to consumer.
2011-12-09 14:16:32,940 Consumer instance (0) exited...
2011-12-09 14:16:32,936 Last call to consumer.
2011-12-09 14:16:32,960 Consumer instance (1) exited...
2011-12-09 14:16:32,945 Last call to consumer.
2011-12-09 14:16:32,960 Last call to consumer.
2011-12-09 14:16:32,994 Consumer instance (3) exited...
2011-12-09 14:16:32,991 Last call to consumer.
2011-12-09 14:16:33,015 Consumer instance (25) exited...
2011-12-09 14:16:32,946 Last call to consumer.
2011-12-09 14:16:33,063 Consumer instance (2) exited...
2011-12-09 14:16:33,023 Last call to consumer.
2011-12-09 14:16:33,080 Consumer instance (4) exited...
2011-12-09 14:16:32,989 Consumer instance (24) exited...
2011-12-09 14:16:33,038 Last call to consumer.
2011-12-09 14:16:33,191 Consumer instance (26) exited...
2011-12-09 14:16:33,085 Last call to consumer.
2011-12-09 14:16:33,242 Consumer instance (5) exited...
2011-12-09 14:16:33,147 Last call to consumer.
2011-12-09 14:16:33,265 Consumer instance (28) exited...
2011-12-09 14:16:33,194 Last call to consumer.
2011-12-09 14:16:33,309 Consumer instance (29) exited...
2011-12-09 14:16:33,101 Last call to consumer.
2011-12-09 14:16:33,336 Consumer instance (6) exited...
2011-12-09 14:16:33,085 Last call to consumer.
2011-12-09 14:16:33,380 Consumer instance (27) exited...
2011-12-09 14:16:33,147 Last call to consumer.
2011-12-09 14:16:33,455 Consumer instance (7) exited...
However, when the same is run from Windows (Python 2.7.2), I get a very long
and strange exception trace which revolves around pickle.py:
Traceback (most recent call last):
File "pc-example.py", line 39, in <module>
pc.start()
File "C:\cygwin\home\jango\workspace\pc\pc\pc.py", line 103, in start
p_proc.start()
File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Python27\lib\multiprocessing\forking.py", line 271, in __init__
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\lib\multiprocessing\forking.py", line 193, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 600, in save_list
self._batch_appends(iter(obj))
File "C:\Python27\lib\pickle.py", line 636, in _batch_appends
save(tmp[0])
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 396, in save_reduce
save(cls)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found as
thread.lock
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 374, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1378, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 858, in load
dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 880, in load_eof
raise EOFError
EOFError
----------
components: Library (Lib)
messages: 149118
nosy: jango
priority: normal
severity: normal
status: open
title: multiprocessing module: Process.start() fails with EOFError:
pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found as
thread.lock
type: behavior
versions: Python 2.6
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue13569>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
