On Tue, 1 Feb 2011, Chris Rebert wrote: > On Tue, Feb 1, 2011 at 12:30 AM, John O'Hagan <m...@johnohagan.com> wrote: > > I'm starting a server process as a subprocess. Startup is slow and > > unpredictable (around 3-10 sec), so I'm reading from its stdout until I > > get a line that tells me it's ready before proceeding, in simplified > > form: > > > > import subprocess > > proc = subprocess.Popen(['server', 'args'], stdout=subprocess.PIPE) > > while proc.stdout.readline() != "Ready.\n": > > pass > > > > Now I can start communicating with the server, but I eventually realised > > that as I'm no longer reading stdout, the pipe buffer will fill up with > > output from the server and before long it blocks and the server stops > > working. > > > > I can't keep reading because that will block - there won't be any more > > output until I send some input, and I don't want it in any case. > > > > To try to fix this I added: > > > > proc.stdout = os.path.devnull > > > > which has the effect of stopping the server from failing, but I'm not > > convinced it's doing what I think it is. If I replace devnull in the > > above line with a real file, it stays empty although I know there is > > more output, which makes me think it hasn't really worked. > > Indeed. proc.stdout is a file, whereas os.devnull is merely a path > string; the assignment is nonsensical type-wise.
My mistake, of course I meant open(os.path.devnull). > > Simply closing stdout also seems to stop the crashes, but doesn't that > > mean it's still being written to, but the writes are just silently > > failing? In > > Based on some quick experimentation, yes, more or less. > > > either case I'm wary of more elusive bugs arising from misdirected > > stdout. > > > > Is it possible to re-assign the stdout of a subprocess after it has > > started? > > I think that's impossible. (Most of Popen's attributes probably should > be read-only properties to clarify that such actions are don't have > the intended effect.) I don't doubt what you say, but attempting to assign it does seem to do something, as it consistently stops the crashes which occur otherwise. What it does, I have no idea. > > What's the right way to read stdout up to a given line, then > > discard the rest? > > I would think calling Popen.communicate() after you've reached the > given line should do the trick. > http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate > Just ignore its return value. However, that does require sending the > input all at once in a single chunk, which it sounds like may not be > feasible in your case; if so, I have no idea how to do it cleanly. Yes, unfortunately I need to send a lot of precisely-timed short strings, and communicate() blocks after the first call. I tried calling stdout.readline() the right number of times after each input, but that seems fiddly and fragile - for example the number of lines of output is not guaranteed and may vary in the case of errors, and also the extra reads had a noticeable effect on latency, which is important in this case. So far my best bet seems to be closing stdin, which doesn't seem very clean, but it does what I want and seems to be just as fast as using stdin=open(os.devnull) in the Popen call in the first place. Thanks, John -- http://mail.python.org/mailman/listinfo/python-list
