On Aug 11, 12:55 pm, EW <ericwoodwo...@gmail.com> wrote: > Hi > > I'm writing a multithreaded app that relies on Queues to move data > between the threads. I'm trying to write my objects in a general way > so that I can reuse them in the future so I need to write them in such > a way that I don't know how many producer and how many consumer > threads I might need. I also might have different consumer threads do > different tasks (for example one might write to a log and one might > write to SQL) so that again means I can't plan for a set ratio of > consumers to producers. So it's unknown. > > So this means that instead of having 1 Queue that all the producers > put to and that all the consumers get from I actually have 1 Queue per > producer thread that the main body sends to the correct type of > consumer thread. So I could get something like this where 3 producer > threads write to 3 different Queues all of which get read by 1 > consumer thread: > > P1 P2 P3 > \ | / > \ | / > C1 > > So producers 1, 2, and 3 all write to individual Queues and consumer 1 > had a list of those Queues and reads them all. The problem I'm having > is that those producer threads can come and go pretty quickly and when > they die I can cleanup the thread with join() but I'm still left with > the Queue. So I could get something like this: > > P1 P3 > \ | / > \ | / > C1 > > So here the P2 thread has ended and gone away but I still have his > Queue lingering. > > So on a thread I can use is_alive() to check status and use join() to > clean up but I don't see any analogous functionality for Queues. How > do I kill them? I thought about putting a suicide message on the > Queue and then C1 would read it and set the variable to None but i'm > not sure setting the variable to None actually makes the Queue go > away. It could just end up sitting in memory unreferenced - and > that's not good. Additionally, I could have any number of consumer > threads reading that Queue so once the first one get the suicide note > the other consumer threads never would. > > I figure there has to be an elegant way for managing my Queues but so > far I can't find it. Any suggestions would be appreciated and thanks > in advance for any help. > > ps Python rocks.
Whoo..the formatting got torn up! My terrible diagrams are even more terrible! Oh well, I think you'll catch my meaning :) -- http://mail.python.org/mailman/listinfo/python-list
