[Twisted-Python] buildProtocol never called
Hello, First let me introduce a piece of software I have written using Twisted: it's called dfd_keeper[1], and is a BSD/pf port of a generic concept I invented called the dynamic firewall daemon[2]. [1] http://www.subspacefield.org/security/dfd_keeper/ [2] http://www.subspacefield.org/security/dfd/ If you like the concept, please promote the idea; I think it's neat but there hasn't exactly been a groundswell of interest in it. In essence, it allows you to netcat or telnet to the daemon process and issue text-like commands that alter your firewall rules. You can think of it as a "shell" for the firewall, and it can do neat things like create firewall rules that expire automatically after a period of time and so on. The problem I'm having is this; I upgraded my firewall from OpenBSD 4.2 or so to OpenBSD 4.5, and now I'm working with newer versions of all the software including Twisted. The program works fine normally, and can run in the background, but if I invoke a daemonize() routine that turns it into a network daemon, it refuses to serve incoming TCP connections. Actually the TCP connection is made, but the software never responds to it. Specifically, my twisted.internet.protocol.Factory instance is created, but buildProtocol is never called. Before I dive deeply into debugging the software, I thought I'd ask if anyone knew off the top of their head any reason why Twisted might not act the same if it was daemonized (no controlling terminal, no stdin/out/err, forked as a background process). Thanks! Travis H. -- Obama Nation | My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email j...@subspacefield.org to get blacklisted. pgpLOVdA1Vtrp.pgp Description: PGP signature ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] buildProtocol never called
On Sat, May 30, 2009 at 06:17:54PM -0500, travis+ml-twis...@subspacefield.org wrote: > The program works fine normally, and can run in the background, but if I > invoke a daemonize() routine that turns it into a network daemon, it > refuses to serve incoming TCP connections. Actually the TCP connection > is made, but the software never responds to it. > > Specifically, my twisted.internet.protocol.Factory instance is created, > but buildProtocol is never called. > > Before I dive deeply into debugging the software, I thought I'd ask if > anyone knew off the top of their head any reason why Twisted might not > act the same if it was daemonized (no controlling terminal, no > stdin/out/err, forked as a background process). Here's some behavior I've gathered through debugging. Note that all of the daemonization occurs before starting the reactor. If I don't close any file descriptors during daemonization, the program works fine. If I close all file descriptors (3, 4, 5, 6, 7) and above, it works, but upon disconnecting, I get the following error message: File "/etc/keeper_screen", line 675, in reactor.run() File "/usr/local/lib/python2.5/site-packages/twisted/internet/base.py", line 1128, in run self.mainLoop() --- --- File "/usr/local/lib/python2.5/site-packages/twisted/internet/base.py", line 1140, in mainLoop self.doIteration(t) File "/usr/local/lib/python2.5/site-packages/twisted/internet/selectreactor.py", line 126, in doSelect self._preenDescriptors() File "/usr/local/lib/python2.5/site-packages/twisted/internet/selectreactor.py", line 88, in _preenDescriptors self._disconnectSelectable(selectable, e, False) File "/usr/local/lib/python2.5/site-packages/twisted/internet/posixbase.py", line 196, in _disconnectSelectable selectable.connectionLost(failure.Failure(why)) File "/usr/local/lib/python2.5/site-packages/twisted/internet/posixbase.py", line 150, in connectionLost os.close(fd) exceptions.OSError: [Errno 9] Bad file descriptor If I close all file descriptors two and above, I get that error message immediately. If I close all file descriptors eight and above, the program works normally. Is there some code in Twisted that expects certain file descriptors to be open? Anyone got a clue what could be happening here? -- Obama Nation | My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email j...@subspacefield.org to get blacklisted. pgpJOapuCdgEv.pgp Description: PGP signature ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] buildProtocol never called
On Sat, May 30, 2009 at 08:26:40PM -0400, Jean-Paul Calderone wrote: > Creating the reactor at all may create a file descriptor which is necessary > for its operation. Closing these will very likely cause problems. This is all before calling any Twisted routines > Also, Twisted has daemonization features already which are known to work > well. ;) Why aren't you using these? Hadn't heard about it. Where's the documentation? -- Obama Nation | My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email j...@subspacefield.org to get blacklisted. pgp4JAWmHNOCY.pgp Description: PGP signature ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] buildProtocol never called
On Sat, May 30, 2009 at 06:17:54PM -0500, travis+ml-twis...@subspacefield.org wrote: > The problem I'm having is this; I upgraded my firewall from OpenBSD > 4.2 or so to OpenBSD 4.5, and now I'm working with newer versions of > all the software including Twisted. > > The program works fine normally, and can run in the background, but if I > invoke a daemonize() routine that turns it into a network daemon, it > refuses to serve incoming TCP connections. Actually the TCP connection > is made, but the software never responds to it. > > Specifically, my twisted.internet.protocol.Factory instance is created, > but buildProtocol is never called. > > Before I dive deeply into debugging the software, I thought I'd ask if > anyone knew off the top of their head any reason why Twisted might not > act the same if it was daemonized (no controlling terminal, no > stdin/out/err, forked as a background process). I ran ktrace/kdump against the binary and it appears that python likes to open file handles to the script and certain modules for file descriptors 5-7 or so (depending on how much you import), and if you close those file descriptors, unusual things happen when python tries to access them. In the end I decided it was not important enough to close all the parent's file descriptors, and it runs fine as a deamon leaving them open. -- Obama Nation | My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email j...@subspacefield.org to get blacklisted. pgpnNEfNHEtwC.pgp Description: PGP signature ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
