----- Original Message -----
From: "Rob Oravec" <[EMAIL PROTECTED]>
Newsgroups: perl.beginners
To: "perl.beginners" <beginners@perl.org>
Sent: Saturday, September 10, 2005 10:15 AM
Subject: Read txt file and use each line as regex and output matches to
file.
Hi All!
Please help this beginner!
I'll try :-)
More than likely there is an easy way to do this and/or I am doing
something stupid but I just don't know how.
Sorry if this is a bit long.
(Scenario)
-Several routers/firewalls log events to a single ($log_file).
-I want to be able to seperate the logs based on router/firewall IP
addresses ($log_source) and dump all
matches to their relevant files for further processing ($date-$ip.log)
-Process the ($ip_file) line by line using each line as a regular
expression, processing the ($log_file).
This way if I add more routers/firewalls to the picture then its just a
case of adding a new IP address to the ($ip_file) and not adding regular
expressions to the script.
(This could be better solved by putting the IP's into a hash.)
But, reg expr will do.
($ip_file)
10.10.10.1
10.10.10.2
10.10.10.3
($log_file)
<snipped>
2005-09-05 00:05:11 Local5.Info 10.10.10.1 %SEC-6-IPACCESSLOGP:
list IN_REF_2 denied tcp xxx.xxx.xxx.xxx(2948) -> xxx.xxx.xxx.xxx(135),
1 packet
2005-09-05 00:05:11 Local5.Info 10.10.10.2 %SEC-6-IPACCESSLOGP:
list IN_REF_2 denied tcp xxx.xxx.xxx.xxx(2949) -> xxx.xxx.xxx.xxx(135),
1 packet
2005-09-05 00:05:11 Local5.Info 10.10.10.3 %SEC-6-IPACCESSLOGP:
list IN_REF_2 denied tcp xxx.xxx.xxx.xxx(2973) -> xxx.xxx.xxx.xxx(135),
1 packet
I'm assuming that there's 1 record per line in $log_file (and not spread
over 3 lines like here)
My result/problem so far:
I think I read the ($ip_file) correctly and then open the ($log_file) to
search through it based on each line in the ($ip_file).
When it has processed the ($log_file) and dumped the results to the
relavant files ($date-$ip.log) it only prints out the last matching line
for each ($log_source) but need it to print all the matches.
The short answer is: it only writes the last line because every time you
open the OUT file for writing, you clear the file of the previous writes.
Solution is to open it in append mode, like:
open(OUT, ">>$log_new") || die "Could not open log_file";
------------------------------------------------------------start-script
#!/usr/bin/perl
Always include the lines:
use strict;
use warnings;
so that perl can catch and tell you of any problems.
It will then be necessary to declare your variables with 'my', as in:
my ($mday,$mon,$year) = (localtime)[3..5];
Anyone who later has to maintain your code, perhaps yourself, will be
thankful that you used strict and warnings :-)
($mday,$mon,$year) = (localtime)[3..5];
$date = sprintf "%d%02d%02d", $year+1900, $mon+1, $mday;
($sec,$min,$hour)=localtime(time);
$time = sprintf "%02d:%02d:%02d", $hour, $min, $sec;
$log_dir = "/home/net1/log";
$rep_dir = "/home/net1/Apache2/htdocs/router-logs";
$log_file = "$log_dir/$date.log";
$ip_file = "/home/net1/log/ip.txt";
open(IPFILE, "$ip_file") || die "Could not open $ip_file";
foreach (<IPFILE>) {
@lines = <IPFILE>;
What is this line for? It does nothing and I don't see where you use @lines
in your code!
Instead of a foreach loop, you would be better off using a while loop. The
foreach reads the entire file into a list whereas 'while' would read 1 line
at a time.
while (<IPFILE>)
chomp($ip);
($ip) = split;
The above 2 lines should be:
chomp $_;
my $ip = $_;
open (LOGFILE, $log_file) || die "Could not open $log_file";
while (<LOGFILE>) {
($datestamp, $timestamp, $facility, $log_source, $restofline) = split
(/\s+/, $_);
Since you don't need all the other variables in the split above, you could
better say:
my $log_source = (split)[3];
if ($log_source =~ /$ip/) {
$log_new = "$rep_dir/$date-$ip.log";
open(OUT, ">$log_new") || die "Could not open log_file";
print OUT;
close(OUT);
print;
}
}
}
close(IPFILE);
close(LOGFILE);
------------------------------------------------------------stop-script
Comment/s:
The search has to be against the ($log_source) variable as the IP
address may appear more than once in the same line and meaning something
completely different.
Other things I have tried:
I have tried using the append option when opening ($log_new) and works
great on the first go, then obviously it appends the next go round.
If you have any other more effective suggestions.......then
please....hit me!
I hope this makes sense and thanks in advance.
Rob
Lastly, you are opening a file for writing for each matched line in your log
file. It would be better to save your matches in a hash. Then you would only
need to open the OUT file once (for each different IP) instead of opening
and writing for each matched line. Just something to consider.
Hope this helps
Chris
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
