On 17 Apr, 04:22, tgiles <[EMAIL PROTECTED]> wrote: > Hi, All! > > I started back programming Python again after a hiatus of several > years and run into a sticky problem that I can't seem to fix, > regardless of how hard I try- it it starts with tailing a log file. > > Basically, I'm trying to tail a log file and send the contents > elsewhere in the script (here, I call it processor()). My first > iteration below works perfectly fine- as long as the log file itself > (logfile.log) keeps getting written to. > > I have a shell script constantly writes to the logfile.log... If I > happen to kill it off and restart it (overwriting the log file with > more entries) then the python script will stop sending anything at all > out. > > import time, os > > def processor(message,address): > #do something clever here > > #Set the filename and open the file > filename = 'logfile.log' > file = open(filename,'r') > > #Find the size of the file and move to the end > st_results = os.stat(filename) > st_size = st_results[6] > file.seek(st_size) > > while 1: > where = file.tell() > line = file.readline() > if not line: > time.sleep(1) > file.seek(where) > else: > print line, # already has newline > data = line > if not data: > break > else: > processor(data,addr) > print "Sending message '",data,"'....." > > someotherstuffhere() > > === > > This is perfectly normal behavior since the same thing happens when I > do a tail -f on the log file. However, I was hoping to build in a bit > of cleverness in the python script- that it would note that there was > a change in the log file and could compensate for it. > > So, I wrote up a new script that opens the file to begin with, > attempts to do a quick file measurement of the file (to see if it's > suddenly stuck) and then reopen the log file if there's something > dodgy going on. > > However, it's not quite working the way that I really intended it to. > It will either start reading the file from the beginning (instead of > tailing from the end) or just sit there confuzzled until I kill it > off. > > === > > import time, os > > filename = logfile.log > > def processor(message): > # do something clever here > > def checkfile(filename): > file = open(filename,'r') > print "checking file, first pass" > pass1 = os.stat(filename) > pass1_size = pass1[6] > > time.sleep(5) > > print "file check, 2nd pass" > pass2 = os.stat(filename) > pass2_size = pass2[6] > if pass1_size == pass2_size: > print "reopening file" > file.close() > file = open(filename,'r') > else: > print "file is OK" > pass > > while 1: > checkfile(filename) > where = file.tell() > line = file.readline() > print "reading file", where > if not line: > print "sleeping here" > time.sleep(5) > print "seeking file here" > file.seek(where) > else: > # print line, # already has newline > data = line > print "readying line" > if not data: > print "no data, breaking here" > break > else: > print "sending line" > processor(data) > > So, have any thoughts on how to keep a Python script from bugging out > after a tailed file has been refreshed? I'd love to hear any thoughts > you my have on the matter, even if it's of the 'that's the way things > work' variety. > > Cheers, and thanks in advance for any ideas on how to get around the > issue. > > tom
Possibly, restarting the program that writes the log file creates a new file rather than appending to the old one?? I think you should always reopen the file between the first and the second pass of your checkfile function, and then: - if the file has the same size, it is probably the same file (but it would better to check the update time!), so seek to the end of it - otherwise, its a new file, and then start reading it from the beginning To reduce the number of seeks, you could perform checkfile only if for N cycles you did not get any data. Ciao ----- FB -- http://mail.python.org/mailman/listinfo/python-list
