On 05Apr2015 12:20, Daniel Ellis <ellis...@gmail.com> wrote:
I have a small little tool I'd like to make. It essentially takes piped input,
modifies the text in some way, and immediately prints the output. The problem
I'm having is that any output I pipe to the program seems to be buffered,
removing the desired effect.
That depends on the upstream program; it does the buffering. The pipe itself
presents received data downstream immediately.
However, as you've seen, almost every program buffers its standard output if
the output is not a tty; this is automatic in the stdio C library and results
in more fficient use of I/O.
From what I understand, I need to somehow have the input be retrieved via a
pseudoterminal.
This is a rather gross hack, though sometimes all you can do. While some
programs have an option to force unbuffered output, most do not. Attaching
their output to a pty is one way to encourage them to at least line buffer
their output.
However, you should bear in mind that the reason that programs line buffer to a
terminal is that they presume they are in an interactive situation with a
person watching. The program _may_ act differently in other ways as well, such
as asking question it might not otherwise ask in "batch" mode (where it might
cautiously not ask and presume "no").
Also, output sent through a pty is subject to the line discipline in the
terminal; temrinals are funny things with much historical behaviour. At the
least you pobably want your pty in "raw" mode to avoid all sorts of stuff that
can be done to your data.
The problem that I'm having is that most examples on the internet seem to
assume I would like to launch a program in a forked pty process, which doesn't
really fit my use case.
Indeed not, but not to worry. You don't need to fork.
I've tried a number of things, but I seem to be unable to get even a basic
understanding of how to use the pty module.
Have you every used a pty from C? Do you know how ptys work? (master side,
slave side, etc).
Here's a piece of code I whipped up just to try to get a feel for what is going
on when I use pty.fork, but it doesn't seem to do what I think it should:
import pty
import os
import sys
pid, fd = pty.fork()
print pid, fd
sys.stdout.flush()
os.read(fd, 1024)
This only seems to print from the parent process.
The documentation for pty.fork says:
Return value is (pid, fd). Note that the child gets pid 0, and the fd is
invalid.
So the child cannot used "fd". It further says that the child has its stdin and
stdout attached to the pty, and that the pty is the child's controlling
terminal (this means it is affected by things like "typing" ^C at the pty,
etc).
I read that I need to do the os.read call for the fork to happen. I've also
tried printing *after* the os.read call.
Don't try to adapt fork-based tutorials to your needs. Understand ptys directly
first.
I realize this does very little to solve my overall goal, but I figure
understanding what is going on is probably a worthwhile first step.
What you probably want to use is pty.openpty() instead. No fork. You will get
back file descriptors for the master and slave sides of the pty. Then you can
use these with the subprocess module to connect your input program. Or,
guessing from your opening sentence, you can write a wrapper script whose whole
purpose is to run a program on a pty.
Regarding terminology: a pseudoterminal (pty) is a device that looks like a
traditional serial terminal. All terminal emulators like xterm use one, and so
do other programs presenting a terminal session such as the sshd process
handling an interactive remote login.
When you call pty.openpty() you are handed two file descriptors: one for the
master side of the pty and one for the slave side. The slave side is the side
that looks like a terminal, and is what a typical use would connect a child
process to. The master side is the other side of the pty. When a program writes
to the "slave" side, the output is available for read on the master side, much
like a pipe. When a program writes to the master side, the output is available
for read on the slave side, _as_ _if_ _typed_ at the terminal.
A pty is not necessarily going to solve your problem unless you can get your
input via the pty. From the sounds of it you're in this situation:
command-generating-output | your-program
such that your input is attached to a pipe, and because
"command-generating-output" is attached to a pipe it is block buffering its
output, hence your problem.
You can't undo that situation after the fact.
To solve your problem via a pty you need to contrive to set up
"command-generating-output" already attached to a pty. One way to do that is
for "your-program" to open a pty and itself invoke "command-generating-output"
with its output via the pty, which is why so many tutorials suppose a "fork"
situation.
One typical away to do that is to pass the "command-generating-output" command
name and args to your program, eg:
your-program command-generating-output [args...]
Then your main program can gather that up:
import sys
command_generating_output = sys.argv[1:]
Then you can call pty.openpty(), and then use the slave file descriptor with
subprocess.Popen to invoke command_generating_output. Thus the generating
command will be talking to you via a pty instead of a pipe.
Cheers,
Cameron Simpson <c...@zip.com.au>
It is interesting to think of the great blaze of heaven that we winnow
down to animal shapes and kitchen tools. - Don DeLillo
--
https://mail.python.org/mailman/listinfo/python-list
