Hello all.
I am trying to read a log file then look for a website and score. If that
website has a score >100, take it and add it to a "Check me" list.
1. I don't want to "recheck" so I have a list of checked sites that I want
to verify with.
2. I don't want duplicates added to the "check me" list.
Below is what I have. I have 1 above working but I cannot get the
duplicates from printing to the list. The sub testfordup at the bottom will
print the proper information but if it is in the log again it will print it
again.
Also, is the a better way of not printing a line if the line exists in the
file?
I am also getting:
Use of uninitialized value in hash element at line 60.
and
Use of uninitialized value in regexp compilation at line 44, <AB_EX_FILE>
line 14.
I think this is because the text is www.site.com an the "." is not being
read as text in the file.
Any help would be greatly appreciated.
#!/usr/bin/perl -w
use strict;
my $ab_file = 'autobannedsitelist1';
my $ab_ex_file = 'autobannedsitelistexceptlst';
my $log_file = 'access.log';
open ( LOG_FILE, "-|", "tail -n 100000 $log_file" ) or die "No log file
exists.\n $! \n";
{
# Begin parsing of log data
my $pass = 0;
my $dup = 0;
foreach my $logfile_line (<LOG_FILE>) {
$logfile_line =~ s/GET//;
if ($logfile_line =~ m/Weighted/ ){
my @logfile_fields = split /\s+/, $logfile_line, 6;
my $site = (split /\//, ($logfile_fields[4]))[2]; # get the domain
name
my $sitemain = (split '\.', $site)[-2]; # get the site name without
.com
my $sitemain2 = (split '\.', $site)[-3]; # get the site name without
.co.ku
my $points = (split ' ',(split ':',
(substr($logfile_fields[5],35,10)))[1])[0];
if ($points > 100){
$pass = &abextest($sitemain,$sitemain2);
if ( $pass eq 0 ){
&testfordup($site);
}
}
}
}
close (LOG_FILE);
}
sub abextest {
my ($sitemain,$sitemain2)[EMAIL PROTECTED];
my $pass = 0;
open ( AB_EX_FILE, "<", $ab_ex_file ) or die "Can't write
AUTOBANNED_EX_FILE: $!";
foreach my $line (<AB_EX_FILE>){
if (($line =~ m/$sitemain/i ) || ( $line =~
m/$sitemain2/i )) {
$pass = 1;
}
}
close (AB_EX_FILE);
return $pass;
}
sub testfordup {
my %seen;
open AB_FILE, "< $ab_file" or die "Can't read AUTOBANNED_FILE: $!";
while (<AB_FILE>) { $seen{$_} = 1 } # build the hash of seen lines
close AB_FILE;
my ($site) = @_;
open AB_FILE, ">> $ab_file" or die "Can't append AUTOBANNED_FILE:
$!";
print AB_FILE "$site\n" if not ($seen{$_}++) ;
close (AB_FILE);
return;
}
Ryan
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
