MonkeeSage <[EMAIL PROTECTED]> wrote:
> Couple of things. You should use poll() on the Popen instance, and
> should check it explicitly against None (since a 0 return code,
> meaning exit successfully, will be treated as a false condition the
> same as None). Also, in your second example, you block the program
> when you call readlines on the pipe, since readlines blocks until it
> reaches eof (i.e., until pipe closes stdout, i.e., process is
> complete). Oh, and you don't have to split the input to the args
> option yourself, you can just pass a string.
Though passing an array is good practice if you want to avoid passing
user data through the shell.
> So, putting it all together, you want something like:
>
> import subprocess, time
>
> cmd = "cat somefile"
> proc = subprocess.Popen(args=cmd, shell=True,
> stdout=subprocess.PIPE, stdin=subprocess.PIPE,
> stderr=subprocess.STDOUT, close_fds=True)
>
> while 1:
> time.sleep(1)
> if proc.poll() != None:
> break
> else:
> print "waiting on child..."
>
> print "returncode =", proc.returncode
This works fine unless the command generates a lot of output (more
than 64k on linux) when the output pipe will fill up and the process
will block until it is emptied.
If you run the below with `seq 10000` then it works fine but as
written the subprocess will block forever writing its output pipe
(under linux 2.6.23).
#------------------------------------------------------------
import subprocess, time
cmd = """
for i in `seq 20000`; do
echo $i
done
exit 42
"""
proc = subprocess.Popen(args=cmd, shell=True,
stdout=subprocess.PIPE, stdin=subprocess.PIPE,
stderr=subprocess.STDOUT, close_fds=True)
while 1:
time.sleep(1)
if proc.poll() != None:
break
else:
print "waiting on child..."
print "returncode =", proc.returncode
lines = 0
total = 0
for line in proc.stdout:
lines += 1
total += len(line)
print "Received %d lines of %d bytes total" % (lines, total)
#------------------------------------------------------------
So you do need to read stuff from your subprocess, but there isn't a
way in the standard library to do that without potentially blocking.
There are a few solutions
1) use the python expect module (not windows)
http://pexpect.sourceforge.net/
2) set your file descriptors non blocking. The following recipe shows
a cross platform module to do it.
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554
Or just do it with the fcntl module
3) Use a thread to read stuff from your subprocess and allow it to
block on proc.stdout.read()
Here is an example of 2)
#------------------------------------------------------------
import subprocess, time, os
from fcntl import fcntl, F_GETFL, F_SETFL
from errno import EAGAIN
cmd = """
for i in `seq 100000`; do
echo $i
done
exit 42
"""
proc = subprocess.Popen(args=cmd, shell=True,
stdout=subprocess.PIPE, stdin=subprocess.PIPE,
stderr=subprocess.STDOUT, close_fds=True)
# Set non blocking (unix only)
fcntl(proc.stdout, F_SETFL, fcntl(proc.stdout, F_GETFL) | os.O_NONBLOCK)
def read_all(fd):
out = ""
while 1:
try:
bytes = fd.read(4096)
except IOError, e:
if e[0] != EAGAIN:
raise
break
if not bytes:
break
out += bytes
return out
rx = ""
while 1:
time.sleep(1)
if proc.poll() != None:
break
else:
print "waiting on child..."
rx += read_all(proc.stdout)
rx += read_all(proc.stdout)
print "returncode =", proc.returncode
lines = 0
total = 0
for line in rx.split("\n"):
lines += 1
total += len(line)
print "Received %d lines of %d bytes total" % (lines, total)
#------------------------------------------------------------
Which runs like this on my machine
$ python subprocess-shell-nb.py
waiting on child...
waiting on child...
waiting on child...
waiting on child...
waiting on child...
waiting on child...
waiting on child...
waiting on child...
returncode = 42
Received 100001 lines of 488895 bytes total
--
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
