Queue limitations?

[mateom]

Hello,

I'm using Queue to send images from one thread to another, and some of
the images are not appearing in the consumer threadmaybe 1 in 3
arrive. I've tried passing the image data in both string form and as a
PIL Image object, with the same result.

It does work, however, if I use zlib to compress the image string
before passing it to Queue and then decompress it in the consumer
thread. So, my question: Does Queue have some capacity limitation?
(Uncompressed, my images are 786432 long... 512x512x3)

Here's a bit of the code:

# Producer 
img = rayCaster.ReadTexture()
iq.put(img)
#iq.put(zlib.compress(img)) #this works fine

# End producer

# Consumer 
class imageQueue(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)
self.filenum = 0
self.theQueue = Queue.Queue(-1)

def run(self):
while 1:
# for testing, do something simple with images
img = self.theQueue.get()
imgt = Image.frombuffer('RGB',(512,512), img)
#imgt = Image.frombuffer('RGB',(512,512),
zlib.decompress(img)) #this one works
imgt.save( "./imgs_out/%i.png" % self.filenum, "PNG")
self.filenum += 1

def SetQueue(self, q_):
self.theQueue = q_

Thanks,
Matt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Queue limitations?

[mateom]

> the queue holds references to the images, not the images themselves,
> so the size should be completely irrelevant.I use one instance of imageQueue.

hmmm.. true. And it also fails when I use PIL Image objects instead of
arrays. Any idea why compressing the string helps?

I'm using one instance of imageQueue.

-Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Queue limitations?

[mateom]

I should be able to add to the queue as fast as I want, right?  I tried
adding time.sleep(.05) right after put(image) in the producer, and that
fixes it.  There is only one thread producing and one thread consuming.

Thanks for the help.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Queue limitations?

[mateom]

I am creating the images by reading from texture memory using
glGetTexImage(). As an experiment, I tried calling glGetTexImage() only
once and letting imagQueue use that same reference over and over, but
the images were all the same.

So, this leads me to believe that the references returned by
glGetTexImage() are to unique buffers, right? ( documentation for
glGetTexImage() doesn't seem to disagree )

Perhaps, as an OpenGL program, something strange is happening with the
references when my producer method goes out of scope. I guess for now I
can be content to just pass immutable types across the thread boundary.

-- 
http://mail.python.org/mailman/listinfo/python-list