On Mon, Nov 23, 2015 at 2:18 PM, Israel Brewster <isr...@ravnalaska.net> wrote:
> Of course, that last step could be interesting - implementing the block in > such a way as to not tie up the processor, while still getting the data "as > soon" as it is available. Unless there is some sort of built-in > notification system I could use for that? I.e. the thread would "subscribe" > to a notification based on its tag, and then wait for notification. When > the master processing thread receives data with said tag, it adds it to the > dictionary and "publishes" a notification to that tag. Or perhaps the > notification itself could contain the payload? There are a few ways I could see handling this, without having the threads spinning and consuming CPU: 1. Don't worry about having the follow-up code run in the same thread, and use a simple callback. This callback could be dispatched to a thread via a work queue, however you may not get the same thread as the one that made the request. This is probably the most efficient method to use, as the threads can continue doing other work while waiting for a reply, rather than blocking. It does make it harder to maintain state between the pre- and post-request functions, however. 2. Have a single, global, event variable that wakes all threads waiting on a reply, each of which then checks to see if the reply is for it, or goes back to sleep. This is good if most of the time, only a few threads will be waiting for a reply, and checking if the correct reply came in is cheap. This is probably good enough, unless you have a LOT of threads (hundreds). 3. Have an event per thread. This will use less CPU than the second option, however does require more memory and OS resources, and so will not be viable for huge numbers of threads, though if you hit the limit, you are probably using threads wrong. 4. Have an event per request. This is only better than #3 if a single thread may make multiple requests at once, and can do useful work when any of them get a reply back (if they need all, it will make no difference). Generally, I would use option #1 or #2. Option 2 has the advantage of making it easy to write the functions that use the functionality, while option 1 will generally use fewer resources, and allows threads to continue to be used while waiting for replies. How much of a benefit that is depends on exactly what you are doing. Option #4 would probably be better implemented using option #1 in all cases to avoid problems with running out of OS memory - threading features generally require more limited OS resources than memory. Option #3 will also often run into the same issues as option #4 in the cases it will provide any benefit over option #2. Chris -- https://mail.python.org/mailman/listinfo/python-list
