On Wed, Apr 6, 2011 at 2:20 AM, Pierre GM <pierregmc...@gmail.com> wrote:
> All, > > I need to run a third-party binary from a python script and retrieve > its output (and its error messages). I use something like > >>> process = subprocess.Popen(options, stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > >>> (info_out, info_err) = process.communicate() > That works fine, except that the third-party binary in question > doesn't behave very nicely and tend to segfaults without returning any > error. In that case, `process.communicate` hangs for ever. > > I thought about calling a `threading.Timer` that would call > `process.terminate` if `process.wait` doesn't return after a given > time... But it's not really a solution: the process in question can > sometimes take a long time to run, and I wouldn't want to kill a > process still running. > I also thought about polling every x s and stopping when the result of > a subprocess.Popen(["ps","-p",str(initialprocess.pid)], > stdout=subprocess.PIPE) becomes only the header line, but my script > needs to run on Windows as well (and no ps over there)... > > Any suggestion welcome, > Thx in advance > P. > -- > http://mail.python.org/mailman/listinfo/python-list > I'm having a difficult time getting something to hang just from a segfault alone - my test code is below in shar format. Are you pretty sure the process is segfaulting (usually verifiable with strace/truss/par/trace/etc)? Based on what you've described, it sounds like there could be a buffering issue. Note that Popen objects support poll() and wait() methods. If you're truly not sending anything to the subprocess' stdin (I suppose your process could have had something redirected into it though), I doubt you'd need to worry about deadlocks much, so you probably could just read in a loop with select (where the select has a timeout) and a Popen.poll() #!/bin/sh # This is a shell archive (produced by GNU sharutils 4.9). # To extract the files from this archive, save it to some FILE, remove # everything before the `#!/bin/sh' line above, then type `sh FILE'. # lock_dir=_sh09253 # Made on 2011-04-06 17:20 PDT by <dstromberg@benchbox>. # Source directory was `/home/dstromberg/segfault-subprocess'. # # Existing files will *not* be overwritten, unless `-c' is specified. # # This shar contains: # length mode name # ------ ---------- ------------------------------------------ # 64 -rw-r--r-- c.c # 37 -rw-r--r-- Makefile # 197 -rwxr-xr-x p # MD5SUM=${MD5SUM-md5sum} f=`${MD5SUM} --version | egrep '^md5sum .*(core|text)utils'` test -n "${f}" && md5check=true || md5check=false ${md5check} || \ echo 'Note: not verifying md5sums. Consider installing GNU coreutils.' save_IFS="${IFS}" IFS="${IFS}:" gettext_dir=FAILED locale_dir=FAILED first_param="$1" for dir in $PATH do if test "$gettext_dir" = FAILED && test -f $dir/gettext \ && ($dir/gettext --version >/dev/null 2>&1) then case `$dir/gettext --version 2>&1 | sed 1q` in *GNU*) gettext_dir=$dir ;; esac fi if test "$locale_dir" = FAILED && test -f $dir/shar \ && ($dir/shar --print-text-domain-dir >/dev/null 2>&1) then locale_dir=`$dir/shar --print-text-domain-dir` fi done IFS="$save_IFS" if test "$locale_dir" = FAILED || test "$gettext_dir" = FAILED then echo=echo else TEXTDOMAINDIR=$locale_dir export TEXTDOMAINDIR TEXTDOMAIN=sharutils export TEXTDOMAIN echo="$gettext_dir/gettext -s" fi if (echo "testing\c"; echo 1,2,3) | grep c >/dev/null then if (echo -n test; echo 1,2,3) | grep n >/dev/null then shar_n= shar_c=' ' else shar_n=-n shar_c= ; fi else shar_n= shar_c='\c' ; fi f=shar-touch.$$ st1=200112312359.59 st2=123123592001.59 st2tr=123123592001.5 # old SysV 14-char limit st3=1231235901 if touch -am -t ${st1} ${f} >/dev/null 2>&1 && \ test ! -f ${st1} && test -f ${f}; then shar_touch='touch -am -t $1$2$3$4$5$6.$7 "$8"' elif touch -am ${st2} ${f} >/dev/null 2>&1 && \ test ! -f ${st2} && test ! -f ${st2tr} && test -f ${f}; then shar_touch='touch -am $3$4$5$6$1$2.$7 "$8"' elif touch -am ${st3} ${f} >/dev/null 2>&1 && \ test ! -f ${st3} && test -f ${f}; then shar_touch='touch -am $3$4$5$6$2 "$8"' else shar_touch=: echo ${echo} 'WARNING: not restoring timestamps. Consider getting and installing GNU `touch'\'', distributed in GNU coreutils...' echo fi rm -f ${st1} ${st2} ${st2tr} ${st3} ${f} # if test ! -d ${lock_dir} ; then : else ${echo} "lock directory ${lock_dir} exists" exit 1 fi if mkdir ${lock_dir} then ${echo} "x - created lock directory ${lock_dir}." else ${echo} "x - failed to create lock directory ${lock_dir}." exit 1 fi # ============= c.c ============== if test -f 'c.c' && test "$first_param" != -c; then ${echo} "x - SKIPPING c.c (file already exists)" else ${echo} "x - extracting c.c (text)" sed 's/^X//' << 'SHAR_EOF' > 'c.c' && X #include <stdio.h> X main() X { X char *p = NULL; X *p = '\0'; X } X SHAR_EOF (set 20 11 04 06 17 12 05 'c.c' eval "${shar_touch}") && \ chmod 0644 'c.c' if test $? -ne 0 then ${echo} "restore of c.c failed" fi if ${md5check} then ( ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'c.c': 'MD5 check failed' ) << \SHAR_EOF bd21e6670ae7a9c711644290c45dc399 c.c SHAR_EOF else test `LC_ALL=C wc -c < 'c.c'` -ne 64 && \ ${echo} "restoration warning: size of 'c.c' is not 64" fi fi # ============= Makefile ============== if test -f 'Makefile' && test "$first_param" != -c; then ${echo} "x - SKIPPING Makefile (file already exists)" else ${echo} "x - extracting Makefile (text)" sed 's/^X//' << 'SHAR_EOF' > 'Makefile' && X go: c X ./p X c: c.c X $(CC) -o c c.c X SHAR_EOF (set 20 11 04 06 17 12 48 'Makefile' eval "${shar_touch}") && \ chmod 0644 'Makefile' if test $? -ne 0 then ${echo} "restore of Makefile failed" fi if ${md5check} then ( ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'Makefile': 'MD5 check failed' ) << \SHAR_EOF 9d831f5d17a790719720363996179f44 Makefile SHAR_EOF else test `LC_ALL=C wc -c < 'Makefile'` -ne 37 && \ ${echo} "restoration warning: size of 'Makefile' is not 37" fi fi # ============= p ============== if test -f 'p' && test "$first_param" != -c; then ${echo} "x - SKIPPING p (file already exists)" else ${echo} "x - extracting p (text)" sed 's/^X//' << 'SHAR_EOF' > 'p' && #!/usr/bin/python X import subprocess X process = subprocess.Popen('./c', stdout=subprocess.PIPE, stderr=subprocess.PIPE) (info_out, info_err) = process.communicate('foo') print(process.returncode) X SHAR_EOF (set 20 11 04 06 17 19 15 'p' eval "${shar_touch}") && \ chmod 0755 'p' if test $? -ne 0 then ${echo} "restore of p failed" fi if ${md5check} then ( ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'p': 'MD5 check failed' ) << \SHAR_EOF 4484b44e31ea9c233139f76818aa2183 p SHAR_EOF else test `LC_ALL=C wc -c < 'p'` -ne 197 && \ ${echo} "restoration warning: size of 'p' is not 197" fi fi if rm -fr ${lock_dir} then ${echo} "x - removed lock directory ${lock_dir}." else ${echo} "x - failed to remove lock directory ${lock_dir}." exit 1 fi exit 0
-- http://mail.python.org/mailman/listinfo/python-list
