[issue4999] multiprocessing.Queue does not order objects

Change by Irit Katriel : -- resolution: -> out of date stage: -> resolved status: pending -> closed ___ Python tracker ___ ___ Pyth

[issue4999] multiprocessing.Queue does not order objects

Irit Katriel added the comment: I see the documentation mentions it now (in the second gray box here: https://docs.python.org/3.8/library/multiprocessing.html#pipes-and-queues ) and it also suggests to use a shared queue if this is an issue. Any objections to closing this issue? --

[issue4999] multiprocessing.Queue does not order objects

Changes by Jean-Paul Calderone : -- nosy: -exarkun ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue4999] multiprocessing.Queue does not order objects

Changes by Ned Deily : -- nosy: +sbt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mai

[issue4999] multiprocessing.Queue does not order objects

Changes by Stefan Krah : -- nosy: -skrah ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.or

[issue4999] multiprocessing.Queue does not order objects

Georg Brandl added the comment: Are you sure this needs to be a warning? We try to use them very sparingly. (Also note the 3-space indent for reStructuredText markup.) -- nosy: +georg.brandl ___ Python tracker __

[issue4999] multiprocessing.Queue does not order objects

Ask Solem added the comment: Updated doc patch -- nosy: +asksol Added file: http://bugs.python.org/file19350/issue-4999.diff ___ Python tracker ___ __

[issue4999] multiprocessing.Queue does not order objects

Stefan Praszalowicz added the comment: I just got surprised by this, and I agree that updating the doc would be nice, because as of now, it states quite explicitly that "the Queue and JoinableQueue types are multi-producer, multi-consumer FIFO queues". -- nosy: +Stefan.P type: -> beh

[issue4999] multiprocessing.Queue does not order objects

Stefan Krah added the comment: I think it would be nice to update the documentation if this isn't resolved yet. The patch adds a warning that FIFO behavior is not guaranteed. -- keywords: +patch nosy: +skrah Added file: http://bugs.python.org/file16276/warn_fifo.patch _

[issue4999] multiprocessing.Queue does not order objects

Changes by Gabriel Genellina : -- nosy: +gagenellina ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue4999] multiprocessing.Queue does not order objects

Jesse Noller added the comment: On a put to the queue, the object is appended onto a deque (self._buffer) - this buffer is managed by a thread (_start_thread) which is handed the _feed method as the target. I suspect the bug is actually in the _feed method. __

[issue4999] multiprocessing.Queue does not order objects

Jesse Noller added the comment: As a note to myself: adding a simple print to the multiprocessing.Queue put method, I can see the calls occurring from the children in order, for example: obj: 0.025193 [proc1] Got lock obj: 0.227725 [proc1] Released lock obj: 0.228401 [proc2] Got lock obj:

[issue4999] multiprocessing.Queue does not order objects

Jean-Paul Calderone added the comment: Jesse, can you explain the cause of the bug? Maybe that will inspire someone to come up with an idea for a fix. -- nosy: +exarkun ___ Python tracker _

[issue4999] multiprocessing.Queue does not order objects

Antoine Pitrou added the comment: That's why I was suggesting to leave Queue as it is and provide a separate PriorityQueue. With a PriorityQueue the user is free to put (timestamp, object) tuples in it if he wants a strict chronologic ordering. He can also use sequence numbers instead of timesta

[issue4999] multiprocessing.Queue does not order objects

Jesse Noller added the comment: I agree, at very least it should provide the same default behavior that Queue does, however, it is not obvious to me how to resolve this in mp.Queue ___ Python tracker _

[issue4999] multiprocessing.Queue does not order objects

Raymond Hettinger added the comment: The Queue module design should be followed as closely as possible. It provides FIFO order as the default, with options for LIFO and priority order. -- nosy: +rhettinger ___ Python tracker

[issue4999] multiprocessing.Queue does not order objects

Antoine Pitrou added the comment: Rather than providing strict ordering of Queue input/output, perhaps a separate or derived class (PriorityQueue?) should be provided just for that matter (heapq-based?). -- nosy: +pitrou ___ Python tracker

[issue4999] multiprocessing.Queue does not order objects

Changes by Jesse Noller : -- priority: -> normal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue4999] multiprocessing.Queue does not order objects

Changes by Jesse Noller : -- assignee: -> jnoller nosy: +jnoller ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue4999] multiprocessing.Queue does not order objects

Frédéric Sagnes added the comment: Tested using Python 2.6.1 on Mac OS 10.5 and Linux 2.6.26 ___ Python tracker ___ ___ Python-bugs-list mailin

[issue4999] multiprocessing.Queue does not order objects

New submission from Frédéric Sagnes : Objects contained in a multiprocessing.Queue object are not comming out of the queue in the same order as they went in. For instance, if I put in object1, object2 and object3 in this very time sequence from multiple processes, they can end up comming out of t