On May 9, 12:12 am, John Nagle <[EMAIL PROTECTED]> wrote:
> skunkwerk wrote:
> > i'm getting the wrong output for the 'title' attributes for this
> > data. the queue holds a data structure (item name, position, and list
> > to store results in). each thread takes in an item name and queries a
> > database for various attributes. from the debug statements the item
> > names are being retrieved correctly, but the attributes returned are
> > those of other items in the queue - not its own item. however, the
> > model variable is not a global variable... so i'm not sure what's
> > wrong.
>
> > i've declared a bunch of workerthreads(100) and a queue into which
> > new requests are inserted, like so:
>
> > queue = Queue.Queue(0)
> > WORKERS=100
> > for i in range(WORKERS):
> > thread = SDBThread(queue)
> > thread.setDaemon(True)
> > thread.start()
>
> > the thread:
>
> > class SimpleDBThread ( threading.Thread ):
> > def __init__ ( self, queue ):
> > self.__queue = queue
> > threading.Thread.__init__ ( self )
> > def run ( self ):
> > while 1:
> > item = self.__queue.get()
> > if item!=None:
> > model = domain.get_item(item[0])
> > logger.debug('sdbthread item:'+item[0])
> > title = model['title']
> > scraped = model['scraped']
> > logger.debug("sdbthread title:"+title)
>
> > any suggestions?
> > thanks
>
> Hm. We don't have enough code here to see what's wrong.
> For one thing, we're not seeing how items get put on the queue. The
> trouble might be at the "put" end.
>
> Make sure that "model", "item", "title", and "scraped" are not globals.
> Remember, any assignment to them in a global context makes them a global.
>
> You should never get "None" from the queue unless you put a "None"
> on the queue. "get()" blocks until there's work to do.
>
> John Nagle
thanks John, Gabriel,
here's the 'put' side of the requests:
def prepSDBSearch(results):
modelList = [0]
counter=1
for result in results:
data = [result.item, counter, modelList]
queue.put(data)
counter+=1
while modelList[0] < len(results):
print 'waiting...'#wait for them to come home
modelList.pop(0)#now remove '0'
return modelList
responses to your follow ups:
1) 'item' in the threads is a list that corresponds to the 'data'
list in the above function. it's not global, and the initial values
seem ok, but i'm not sure if every time i pass in data to the queue it
passes in the same memory address or declares a new 'data' list (which
I guess is what I want)
2) john, i don't think any of the variables you mentioned are
global. the 'none' check was just for extra safety.
3) the first item in the modelList is a counter that keeps track of
the number of threads for this call that have completed - is there any
better way of doing this?
thanks again
--
http://mail.python.org/mailman/listinfo/python-list
