Python open a named pipe == hanging?

[Rochester]

Hi,

 I just found out that the general open file mechanism doesn't work
 for named pipes (fifo).  Say I wrote something like this and it
 simply hangs python:

#!/usr/bin/python

import os

os.mkfifo('my_fifo')

open('my_fifo', 'r+').write('some strings.')
x = os.popen('cat my_fifo').read()

print x


 I know I could use a tempfile instead of a fifo in this very
 simple case, I just want to know is there a standard way of
 handling fifos withing python.  Especially the non-trivial case
 when I want to call a diff like system program which takes two
 files as input.  Say I have two python string objects A and B, I
 want to call diff to see what is the different between those two
 strings, and return the finding as a string obj C back to python.
 This certainly can be done in this way:

open('tmpfile1', 'w').write(A)
open('tmpfile2', 'w').write(B)
C = os.popen('diff tmpfile1 tmpfile2').read()

 But that's kinda awkward isn't it? :-) The Bash way of doing this
 would be (suppose A is the stdout of prog2, B is the stdout of
 prog3):

diff <(prog2) <(prog3) > C

What is the best way of doing this in Python?

Thank you!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python open a named pipe == hanging?

[Rochester]

Thank you for your advise.  So, it turns out that fifos are quite useless  
in Python programming then, which is quite disappointing to me :-(

I am not saying that I _have to_ use fifo, afterall it is a rather odd  
thingy not in fasion since the last iceage... I am just disappointed by  
the fact that the old plain Bash seems to excel Python in this special  
aspect.

I am new to Python and much more comfortable in Bash programming.  A  
simple Bash script like this would take the advantage of a fifo, hence  
reduce the overhead of unneccesarry temporary files creation:

#!/bin/bash

mkfifo my_fifo
echo "this is a string in my fifo!" > my_fifo &
cat my_fifo
rm my_fifo

Isn't it neat?

Anyway, I think every scripting language has its pros and cons.  Bash is  
probably more flexible in dealing with fifos and multiway pipes (through  
the magic menchanism of process substitution).

Thank you!


On Thu, 03 Aug 2006 22:13:56 -0400, Alex Martelli <[EMAIL PROTECTED]> wrote:
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python open a named pipe == hanging?

[Rochester]

Thanks Alex, now I think I understand much better the fifo/pipe mechanism  
and how Python treats them.

For those who are interested, I would like to restate the problem I was  
tring to solve and a working solution (inspired by Alex Martelli's code),  
feel free to criticize it:

The problem:

I have an external program which takes two matrices (two text files) as  
input, and produces an output to stdout, you can take diff as an example.   
I want to call this program from within Python, without writing any  
temporary file (for the performance reason).

In Bash, this task would be best written as:

#!/bin/bash

diff <(step1) <(step2) | step3

Where step1, step2 and step3 have to be independent external programs.

Now in Python, since there is no exact equivalence of <() magic a.k.a.  
process substitution, I figured taht the best solution should be to create  
a pair of fifos and do something like this:

#!/bin/bash

mkfifo fifo1 fifo2
step1 > fifo1 &
step2 > fifo2 &
diff step1 step2 | step3

And a working Python equivalent code is:

#!/usr/bin/python

import os

# do step1 and step2 in Python, say we end up with something like these:

s1 = "some string\n second line."  # results from step1
s2 = "some string\n a different line." # results from step2

os.mkfifo('fifo1')
os.mkfifo('fifo2')

op = os.popen(' '.join(['diff', 'fifo1', 'fifo2'])) # this step is crucial!
print >> open('fifo1', 'w'), s1
print >> open('fifo2', 'w'), s2

os.unlink('fifo1')
os.unlink('fifo2')

x = op.read()

# Now do something about x

print x

P.S.: process substitution is a Bash hack which uses /dev/fd/ to send  
(usually more than one) processes output to a program which takes (more  
than one) filename as input.  Heuristically speaking, this is like a  
multiway pipe.  You can find more about this mechanism here:

http://www.tldp.org/LDP/abs/html/process-sub.html
-- 
http://mail.python.org/mailman/listinfo/python-list