On Wednesday 05 May 2010, Rob Coops wrote: > > A file never starts life being huge but certainly logs tend to grow, and > they are not always kept in check properly so assume they will be massive > (I've seen flat text logs that grew by as much as +1GB per day) assuming > that the file will always be relatively small because it is at this point in > time is very dangerous especially if you have little or no control over that > file your self.
Eventhough what you said does make sense, I don't think it's relevant here. It's not about whether you have the control over the file or not, but what would you do if you want to open a huge file in a Perl program. You can either skip it or you will have to store in a variable and get on with the memory issue. > An if/elsif construction is nice for a few conditions but that will not work > for say 50 of them even at 10 the script will become hard to read, of course > one could construct a hash where the key is the condition and the value is > an array of lines found that match that condition, but this runs into the > same memory problem again. > The only way to do this and not have to worry about memory issues is by > resetting the file handle to the start of the file at the start of the loop, > that way you avoid any memory problems and you avoid having to open and > close the file say 50 times. Yes, this is the right solution if you are going to play with the file contents multiple times, but the only exception is when the file size is huge and I've already talked about this above. > One other thing I would like to remark is that when you are monitoring a log > or anything else just reporting errors is not a good idea you want to report > both the good and the bad. Think of the following situation: Your machines > logs are flooded with errors but due to a configuration error your scheduler > is not starting the monitoring script or the script is started but due to a > permissions problem it cannot access the file or it cannot find a path to > the mail server due to a routing issue etc... you will never know that your > logs are being spammed with errors since if you don't get a mail this means > everything is fine, right? > I have been in a situation where a critical production system had been > screaming that the restoration of its raid array had caused the database on > it to become inconsistent but due to an error in the routing tables the > error mails where being sent out from an incorrect interface and the mail > server could not be reached. It took over 3 months before the system went > down in a catastrophic failure when there was no other solution then to > rebuild the whole database, all of that just because positive news was not > being send out. > So if you want to rely on mails send from the machine to monitor logs make > sure you send a mail even if all it says that there is nothing to report. In a real world scenario, system admins will have to setup a script to send out notifications if something fails. You can't set it up in such a way that it will send the email whatever the result be. You can do it if you are doing it only for a few, but not really feasible when you have got a lot more systems. If you do, you'll end up with receiving lots of emails and you might miss those failure notifications in such cases. So it's better to have the system to send out email only when there's an issue. But then, the issue you mentioned might happen. If the email wasn't delivered due to some reason, you wouldn't know there's an issue. But you have a better solution there as well: Just perform periodic audits so that you can manually confirm everything is fine. For a critical component such as RAID, it's important to have a live monitoring system such as nagios so that you'll never miss out on such things. At the end of day, it comes down to how you manage what you have got in your hands. -- Regards, Akhthar Parvez K http://Tips.SysAdminGUIDE.COM UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity - Dennie Richie -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
