On Apr 29, 4:32 pm, [EMAIL PROTECTED] wrote: > On 27 Apr, 12:27, Terry <[EMAIL PROTECTED]> wrote: > > > > > Hello! > > > I'm trying to implement a message queue among threads using Queue. The > > message queue has two operations: > > PutMsg(id, msg) # this is simple, just combine the id and msg as one > > and put it into the Queue. > > WaitMsg(ids, msg) # this is the hard part > > > WaitMsg will get only msg with certain ids, but this is not possible > > in Queue object, because Queue provides no method to peek into the > > message queue and fetch only matched item. > > > Now I'm using an ugly solution, fetch all the messages and put the not > > used ones back to the queue. But I want a better performance. Is there > > any alternative out there? > > > This is my current solution: > > > def _get_with_ids(self,wait, timeout, ids): > > to = timeout > > msg = None > > saved = [] > > while True: > > start = time.clock() > > msg =self.q.get(wait, to) > > if msg and msg['id'] in ids: > > break; > > # not the expecting message, save it. > > saved.append(msg) > > to = to - (time.clock()-start) > > if to <= 0: > > break > > # put the saved messages back to the queue > > for m in saved: > > self.q.put(m, True) > > return msg > > > br, Terry > > Wy put them back in the queue? > You could have a defaultdict with the id as key and a list of > unprocessed messages with that id as items. > Your _get_by_ids function could first look into the unprocessed > messages for items with that ids and then > look into the queue, putting any unprocessed item in the dictionary, > for later processing. > This should improve the performances, with a little complication of > the method code (but way simpler > that implementing your own priority-based queue). > > Ciao > ----- > FB
Yes, this will improve the performance. And I can see there's a problem in my current implementation. The order of the message might be changed if I put the saved message back to the end of the queue. This may cause some confusion later, though I don't want to depend too much on the message orders. And you remind me one thing -- I need to implement 'priority' for messages, so that the message with highest priority will tend to be fetched first. OMG, this is going to be much more complicated then I have expected. Thanks for your suggestion. And I hope this will also work when I move to stackless. -- http://mail.python.org/mailman/listinfo/python-list
