[issue18277] Queue is empty right after put from the same process/thread

[shwouchk]

New submission from shwouchk:

Consider this:

$ python
Python 2.7.4 (default, Apr 19 2013, 18:28:01) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing as mp
>>> q = mp.Queue()
>>> while True:
   q.put(1)
   q.get_nowait()

Traceback (most recent call last):
  File "", line 3, in 
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 152, in get_nowait
return self.get(False)
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 134, in get
raise Empty
Queue.Empty

I believe that similar behavior could be seen in cPython 2.7.3 with the 
Queue.Queue implementation, but I can't reproduce it now and don't have the old 
version to test. And it is irrelevant anyway since it work "correctly" now.

I think this behavior is counter intuitive and hampers the development of code 
that performs stuff with queues in a generic way and works in both single and 
multi-process environments.

--
components: IO, Interpreter Core
messages: 191566
nosy: shwouchk
priority: normal
severity: normal
status: open
title: Queue is empty right after put from the same process/thread
type: behavior
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue18277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18277] Queue is empty right after put from the same process/thread

[shwouchk]

shwouchk added the comment:

The major difference with the issue you referenced is that the behavior in 
#17985 is clearly stated and warned against in the docs (Queue.Empty being 
inconsistent with Queue.size), whereas this is not.

As for the actual behavior problem: Imagine you build an abstraction atop Queue 
(For example and specifically, a queue iterator). This iterator might return 
None or a similar sentinel if there is nothing on the queue.

Since you find the construct useful and would like to keep uniformity in the 
program, there is a place you use this abstraction in a fairly tight loop to 
pass messages from one part of the program to another part, in the same process 
and thread. Now the logic of the program does not work correctly as it was 
based on the assumption that in a single process Queue would work "as expected".

The problem is not the exception but with having an abstraction built atop 
Queue that works right with respect to program logic both when you use it in 
one thread and in multiple ones.

--

___
Python tracker 
<http://bugs.python.org/issue18277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18277] Queue is empty right after put from the same process/thread

[shwouchk]

shwouchk added the comment:

Richard,

I think you missed my point. First, yes I did do that.

Second ("the point"):

I did this to use the same abstraction that was used extensively for other 
purposes, instead of recreating the same abstraction with a deque as its basis. 
Component reusability is one of the main points of OOP, after all...

And no, an item is not necessarily available - sometimes there is a message and 
sometimes there isn't. But if one was put into the queue, I claim that I should 
be able to rely on it being available right away in the application logic.

--

___
Python tracker 
<http://bugs.python.org/issue18277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18277] Queue is empty right after put from the same process/thread

[shwouchk]

shwouchk added the comment:

Also, of course I did this or I would not have stumbled into this issue...

--

___
Python tracker 
<http://bugs.python.org/issue18277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18277] Queue is empty right after put from the same process/thread

[shwouchk]

shwouchk added the comment:

I agree it might be less efficient, but sometimes it is the price to pay for 
greater generality/simplicity. After all, If I *really* wanted efficiency 
perhaps I would have written everything in C++.

Anyway, thanks!

n.p.

1. "but should not cause any pratical difficulties" <-- you have a typo in 
'pratical' there.
2. What exactly do you mean by "managed" queues in the new addition?

Also, did part #2 of the note come up in other reports? It seemed somewhat 
trivial (can't hurt though)...

Finally, I don't know whether I'm supposed to close the issue, but its a good 
solution as far as I'm concerned.

Thanks again!

--

___
Python tracker 
<http://bugs.python.org/issue18277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com