Hi
I'm writing a multithreaded app that relies on Queues to move data
between the threads. I'm trying to write my objects in a general way
so that I can reuse them in the future so I need to write them in such
a way that I don't know how many producer and how many consumer
threads I might need. I also might have different consumer threads do
different tasks (for example one might write to a log and one might
write to SQL) so that again means I can't plan for a set ratio of
consumers to producers. So it's unknown.
So this means that instead of having 1 Queue that all the producers
put to and that all the consumers get from I actually have 1 Queue per
producer thread that the main body sends to the correct type of
consumer thread. So I could get something like this where 3 producer
threads write to 3 different Queues all of which get read by 1
consumer thread:
P1 P2 P3
\ | /
\ | /
C1
So producers 1, 2, and 3 all write to individual Queues and consumer 1
had a list of those Queues and reads them all. The problem I'm having
is that those producer threads can come and go pretty quickly and when
they die I can cleanup the thread with join() but I'm still left with
the Queue. So I could get something like this:
P1 P3
\ | /
\ | /
C1
So here the P2 thread has ended and gone away but I still have his
Queue lingering.
So on a thread I can use is_alive() to check status and use join() to
clean up but I don't see any analogous functionality for Queues. How
do I kill them? I thought about putting a suicide message on the
Queue and then C1 would read it and set the variable to None but i'm
not sure setting the variable to None actually makes the Queue go
away. It could just end up sitting in memory unreferenced - and
that's not good. Additionally, I could have any number of consumer
threads reading that Queue so once the first one get the suicide note
the other consumer threads never would.
I figure there has to be an elegant way for managing my Queues but so
far I can't find it. Any suggestions would be appreciated and thanks
in advance for any help.
ps Python rocks.
--
http://mail.python.org/mailman/listinfo/python-list
