On Wed, Aug 26, 2009 at 6:41 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>wrote:
> On Wed, Aug 26, 2009 at 4:56 PM, Martin-Louis Bright > <mlbri...@gmail.com>wrote: > >> I would like to write a small daemon that monitors (tails) a server log, >> parses the entries and sends HTTP requests based on some of those entries. I >> would like it if the reading of the log file and the sending of http >> requests were asynchronous. Should I use twisted for this? Or is twisted >> overkill... >> > Twisted would be perfectly appropriate for this! The HTTP requests will > certainly be non-blocking, assuming you use twisted.web.client (or the new, > better thing which I hope will be released before we all get old). > > In fact, if anything, Twisted is under-kill; you'll need to go a bit > further. The one minor issue is that Twisted has no explicit way to do > asynchronous file I/O (because most operating systems provide a bewildering > array of not-really-working systems for non-blocking file I/O, so it would > be challenging for Twisted to do in a way that was useful). There are a > number of ways to emulate it though; for log files (which are fairly low > volume, and almost by definition will immediately be in your filesystem > cache as the parts you want to read become available) you can just do a > LoopingCall() which does .read() on a file that it leaves open to retrieve > the next chunk of data, and that will probably be good enough. In practice > you won't actually block doing this read() because your kernel is just going > to pull the bytes directly out of filesystem cache memory and hand them back > to you. It is of course possible to have to wait for the disk or even the > network, depending on your underlying filesystem. > > I just filed http://twistedmatrix.com/trac/ticket/3983 for a more thorough > solution (mostly in the hopes that somebody will close it as a duplicate and > point to some pre-existing issue I couldn't find through search) so you can > monitor future discussion there if you like. > It would certainly be nice if Twisted supported async file io, but in this case wouldn't a ProcessProtocol around 'tail -f' be a good solution as well? -Cary > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > -- 01100011 01100001 01110010 01111001
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
