On Jun 11, 6:00 am, George Sakkis <[EMAIL PROTECTED]> wrote: > On Jun 11, 1:59 am, Rhamphoryncus <[EMAIL PROTECTED]> wrote: > > > Why not use a normal Queue, put a dummy value (such as None) in when > > you're producer has finished, and have the main thread use the normal > > Thread.join() method on all your child threads? > > I just gave two reasons: > - Concurrency / interactivity. The main thread shouldn't wait for all > one million items to be produced to get to see even one of them.
Then don't wait. The main thread can easily do other work while the producer and consumer threads go about their business. > - Limiting resources. Just like iterating over the lines of a file is > more memory efficient than reading the whole file in memory, getting > each consumed item as it becomes available is more memory efficient > than waiting for all of them to finish. That's why you give Queue a maxsize. Put it at maybe 5 or 10. Enough that the producer can operate in a burst (usually more efficient that switching threads after each item), then the consumer can grab them all in a burst. Then again, you may find it easier to use an event-driven architecture (like Twisted, as others have suggested.) -- http://mail.python.org/mailman/listinfo/python-list
