Josiah Carlson added the comment:

Due to some rumblings over on the mentors list and python-dev, this is now 
getting some love.

Guido has stated that something should make it into the subprocess module for 
3.5 in this email: 
https://groups.google.com/d/msg/dev-python/I6adJLIjNHk/Usrvxe_PVJIJ

His suggested API is:
proc.write_nonblocking(data)
data = proc.read_nonblocking()
data = proc.read_stderr_nonblocking()


I've implemented the API for Windows and Linux, and below are my findings.


On the Linux side of things, everything seems to be working more or less as 
expected, though in writing tests I did need to add an optional argument to 
qcat.py in order to have it flush its stdout to be able to read from the parent 
process. Odd, but whatever.

Also, Richard Oudkerk's claims above about not needing to use fcntl to swap 
flags is not correct. It's necessary to not block on reading, even if select is 
used. Select just guarantees that there is at least 1 byte or a closed handle, 
not that your full read will complete.


On the Windows side of things, my assumptions about WriteFile() were flawed, 
and it seems that the only way to make it actually not block if/when the 
outgoing buffer is full is to create a secondary thread to handle the writing*. 
I've scoured the MSDN docs and there doesn't seem to be an available API for 
interrogating the pipe to determine whether the buffer is full, how full it is, 
or whether the write that you'd like to do will succeed or block.

* This applies even with the use of asyncio. Because the intent for the call is 
to return more or less immediately, a thread still has to be created to handle 
the event loop for IO completion, which means that asyncio can't prevent the 
use of threads and not block without basically integrating an event loop into 
the caller.

Considering that the Windows communicate() method already uses threads to 
handle reading and writing, I don't believe it is a big deal to add it for this 
situation too. Any major objections?

----------

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue1191964>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to