Python and Flask Book

[Bruno Oliveira]

Hello all,

I would like to spread the word about a new Python book I have released, on
how to develop a web application using Flask and deploying it on Heroku.
It's geared more towards beginners or towards anyone who is willing to
learn Python and Flask!

I would be very glad if you know of people, friends, or colleagues who are
interested in learning Python and could spread words about my book:

https://brunooliv.gumroad.com/l/cookpythonbook


Best regards,
Bruno
-- 
https://mail.python.org/mailman/listinfo/python-list

Multiprocessing Queue strange behavior

[Bruno Oliveira]

Hi list,

I recently found a bug in my company's code because of a strange behavior
using multiprocessing.Queue. The following code snippet:

from multiprocessing import Queue

queue = Queue()
queue.put('x')
print queue.get_nowait()

Fails with:

...
  File
"E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
line 153, in getNoWait
return self.get(False)
  File
"E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
line 129, in get
raise Empty
Queue.Empty

Strangely, changing this to:

queue = Queue()
queue.put('x')
time.sleep(0.1) # <<<
print queue.get_nowait()

Works as expected. Using the original snippet changing the import to
threading's Queue also works.

It seems like there's a bug in multiprocessing's Queue implementation.
Opinions?

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing Queue strange behavior

[Bruno Oliveira]

Hi, thanks for the answer.

I thought about that, but the problem is that I found the problem in code
that *was* using the Queue between processes. This code for example fails
around 60% of the time in one of our linux machines (raising an Empty
exception):

from processing import Queue, Process
import time

def Consume(queue):
print queue.get_nowait()

if __name__ == '__main__':
queue = Queue()
queue.put('x')
p = Process(target=Consume, args=(queue,))
p.start()
p.join()

Again, putting a sleep call before starting the consumer process makes the
code work all time. That is the main reason I believe there is a problem in
the Queue code somewhere.

Cheers,


On Wed, Sep 15, 2010 at 5:47 PM, MRAB  wrote:

> On 15/09/2010 21:10, Bruno Oliveira wrote:
>
>> Hi list,
>>
>> I recently found a bug in my company's code because of a strange
>> behavior using multiprocessing.Queue. The following code snippet:
>>
>> from multiprocessing import Queue
>>
>> queue = Queue()
>> queue.put('x')
>> print queue.get_nowait()
>> Fails with:
>>
>> ...
>>   File
>>
>> "E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
>> line 153, in getNoWait
>> return self.get(False)
>>   File
>>
>> "E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
>> line 129, in get
>> raise Empty
>> Queue.Empty
>>
>> Strangely, changing this to:
>>
>> queue = Queue()
>> queue.put('x')
>> time.sleep(0.1) # <<<
>> print queue.get_nowait()
>> Works as expected. Using the original snippet changing the import to
>> threading's Queue also works.
>>
>> It seems like there's a bug in multiprocessing's Queue implementation.
>> Opinions?
>>
>>  I don't think it's a bug as such.
>
> The purpose of the multiprocessing queue is to transfer data between
> different processes, which don't have a shared address space (unlike
> threads, which do).
>
> The transfer involves passing the data between the processes via a
> pipe. This is done in a background thread and takes some time to
> complete, so the data won't appear immediately. It looks like it
> doesn't matter that the putter and the getter happen to be in the same
> process, possibly because no-one expected that someone would use a
> multiprocessing queue within the same process like that, so it doesn't
> check for a shortcut.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list