[issue10886] Unhelpful backtrace for multiprocessing.Queue

A.M. Kuchling added the comment: neologix: did you intend to re-open this ticket when you made your 2013-03-05 comment? It seems to me that you didn't intend to -- your comment doesn't say 're-opening because '. I'll close it again; if you want, please re-open it and just explain why. -

[issue10886] Unhelpful backtrace for multiprocessing.Queue

Charles-François Natali added the comment: For the record, I'm posting thse benchmark numbers here (originally from issue #17025): """ with patch: $ ./python /tmp/multi_queue.py took 0.7945001125335693 seconds with 1 workers took 0.7428359985351562 seconds with 2 workers took 0.7897098064422607

[issue10886] Unhelpful backtrace for multiprocessing.Queue

Changes by Charles-François Natali : -- status: closed -> open superseder: reduce multiprocessing.Queue contention -> ___ Python tracker ___

[issue10886] Unhelpful backtrace for multiprocessing.Queue

Charles-François Natali added the comment: I'm closing, since issue #17025 proposes to do this as part of performance optimization. -- nosy: +neologix status: open -> closed superseder: -> reduce multiprocessing.Queue contention ___ Python tracker

[issue10886] Unhelpful backtrace for multiprocessing.Queue

Antoine Pitrou added the comment: > Changes to the implementation of Connection won't affect whether > Queue.put() raises an error immediately if it gets an unpicklable > argument. Ah, right. Then indeed it won't make a difference. -- ___ Python tra

[issue10886] Unhelpful backtrace for multiprocessing.Queue

sbt added the comment: > This shouldn't be a problem in Python 3.3, where the Connection classes > are reimplemented in pure Python. What should not be a problem? Changes to the implementation of Connection won't affect whether Queue.put() raises an error immediately if it gets an unpicklable

[issue10886] Unhelpful backtrace for multiprocessing.Queue

Antoine Pitrou added the comment: This shouldn't be a problem in Python 3.3, where the Connection classes are reimplemented in pure Python. -- nosy: +pitrou ___ Python tracker

[issue10886] Unhelpful backtrace for multiprocessing.Queue

sbt added the comment: mp_queue_pickle_in_main_thread.patch (against the default branch) fixes the problem by doing the pickling in Queue.put(). It is version of a patch for Issue 8037 (although I believe the behaviour complained about in Issue 8037 is not an actual bug). The patch also has

[issue10886] Unhelpful backtrace for multiprocessing.Queue

New submission from Torsten Landschoff : When trying to send an object via a Queue that can't be pickled, one gets a quite unhelpful traceback: Traceback (most recent call last): File "/usr/lib/python2.6/multiprocessing/queues.py", line 242, in _feed send(obj) PicklingError: Can't pickle