On Fri, 21 Nov 2014, at 19:26, Glyph wrote:
>> On Nov 21, 2014, at 15:13, Peter Westlake
>> <peter.westl...@pobox.com> wrote:
>>
>> I*am*missing something obvious. The file opened by open() immediately
>> goes out of scope. AAAUGH!
>
> So... back to square one? Or is this the solution to your problem? I
> don't entirely follow how this connects..
I didn't explain, sorry. This does indeed solve my problem!
It's a mistake in very nearly the first line of the first real Python
program I wrote. It goes likes this:
devnull = open('/dev/null').fileno()
# 1. Create a Python file object on /dev/null. Its file descriptor is
# the first one available, 3.
# 2. Use it in an expression, and store the FD (3) in the variable
# devnull.
# 3. Lose the last reference to it.
# 4. *The file object is garbage collected and the file is closed.*
# 5. The variable devnull remembers the file descriptor number of this
# short-lived file.
reactor.connectTCP('localhost', 'http', factory)
# 6. Open a socket. Its file number is the first one available, which is
# 3 again.
reactor.callWhenRunning(reactor.spawnProcess, ProcessProtocol(),
'/bin/sleep', args=['/bin/sleep', '1000'], childFDs={0: devnull, 1:
'r', 2:'r'})
# 7. Pass the file on descriptor 3 to the spawned process. But it's the
# socket, not /dev/null!
In the real program the socket is an AMP connection, and the
spawnProcess is a vast tree of processes that build a large and complex
piece of software. Eventually the build process reaches an SSH call, and
for some reason SSH sometimes reads from its stdin. I don't know why,
and I don't know why it always tried to read at precisely the same time
at Twisted. But I do know that it does, thanks to a systemtap probe.
To fix the bug (and it *is* fixed, and passes testing) I had the option
of either saving the file object:
devnull = open('/dev/null') ... spawnProcess(...childFDS{0:
devnull.fileno(), 1: 'r', 2: 'r'}...)
or closing stdin:
spawnProcess(...childFDS{1: 'r', 2: 'r'}...)
The fact that the AMP connection ever worked at all showed that the
build process didn't try to read stdin, so I chose the latter option.
The SSH still works, so I suppose it is using a select() or similar
to read stdin. That would account for it being synchronised with
Twisted: when they are both waiting for data, both of them see it at
once and collide.
I suspect that the reason this went away in 2009 was because a load of
SSH calls were removed (they were to Windows machines, and very
unreliable!), and the reason it came back now was because a load more
were put in. The system was moved behind a firewall, and had to do a lot
of its former work by remote control using SSH.
Again, thank you for all the help. I do like Twisted!
Peter.
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
