[issue4660] multiprocessing.JoinableQueue task_done() issue

Jesse Noller added the comment: I used the protected JoinableQueue put method suggested by Brian. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ _

[issue4660] multiprocessing.JoinableQueue task_done() issue

Jesse Noller added the comment: Fix checked into python trunk with r74326, 26 maint w/ r74327 -- ___ Python tracker ___ ___ Python-bug

[issue4660] multiprocessing.JoinableQueue task_done() issue

Brian added the comment: Cool., let me know if there is anything I can do to help. On Mon, Jun 29, 2009 at 7:46 AM, Jesse Noller wrote: > > Jesse Noller added the comment: > > I'm leaning towards the properly protecting JoinableQueue.put() fix, I'm > not a terribly big fan of removing error c

[issue4660] multiprocessing.JoinableQueue task_done() issue

Jesse Noller added the comment: I'm leaning towards the properly protecting JoinableQueue.put() fix, I'm not a terribly big fan of removing error checking. I'm trying to carve off time this week to beat on my bug queue, so I'm hoping to be able to commit something (once I have docs+tests) thi

[issue4660] multiprocessing.JoinableQueue task_done() issue

Brian added the comment: Filipe, Thanks for the confirmation. While I think the second option (ie properly protecting JoinableQueue.put()) is best, the first option (simply removing the 'task_done() called too many times' check) should be safe (presuming your get() and put() calls actually

[issue4660] multiprocessing.JoinableQueue task_done() issue

Filipe Fernandes added the comment: I ran into the same problem and am greatful to Brian for reporting this as I thought I was loosing my mind. Brian noted that he was running windows and I can confirm that Brian's test case is reproducable on my laptop running: Ubuntu 9.04 python 2.6.2 Altho

[issue4660] multiprocessing.JoinableQueue task_done() issue

Changes by Alvaro : -- nosy: +aloga ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailm

[issue4660] multiprocessing.JoinableQueue task_done() issue

Brian added the comment: Jesse, I am afraid my last post may have confused the issue. As I mentioned in my first post, the problem arises when JoinableQueue.put is preempted between its two lines. Perhaps the easiest way to illustrate this is to exacerbate it by modifying JoinableQueue.put to

[issue4660] multiprocessing.JoinableQueue task_done() issue

Brian added the comment: Hey Jesse, It was good meeting you at Pycon. I don't have anything handy at the moment although, if memory serves, the most trivial of example seemed to illustrate the problem. Basically any situation where a joinable queue would keep bumping up against being empty (ie

[issue4660] multiprocessing.JoinableQueue task_done() issue

Jesse Noller added the comment: Hi Brian - do you have a chunk of code that exacerbates this? I'm having problems reproducing this, and need a test so I can prove out the fix. -- ___ Python tracker ___

[issue4660] multiprocessing.JoinableQueue task_done() issue

Changes by Jesse Noller : -- priority: -> high ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue4660] multiprocessing.JoinableQueue task_done() issue

Brian added the comment: Here are a few stabs at how this might be addressed. 1) As originally suggested. Allow task_done() to block waiting to acquire _unfinished_tasks. This will allow the put() process to resume, release() _unfinished_tasks at which point task_done() will unblock. No

[issue4660] multiprocessing.JoinableQueue task_done() issue

Changes by Benjamin Peterson : -- assignee: -> jnoller nosy: +jnoller ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubsc

[issue4660] multiprocessing.JoinableQueue task_done() issue

Changes by Brian : -- type: -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/

[issue4660] multiprocessing.JoinableQueue task_done() issue

New submission from Brian : Despite carefully matching my get() and task_done() statements I would often trigger "raise ValueError('task_done() called too many times')" in my multiprocessing.JoinableQueue (multiprocessing/queues.py) Looking over the code (and a lot of debug logging), it appear