Re: PDFInfo plugin with SA 3.1.7

[John Rudd]

Johann Spies wrote:
On Thu, Jul 12, 2007 at 11:54:51AM +0200, Robert Schetterer wrote:
Hi, after having good results in the beginning
with pdfinfo ,
no one of the following pdf spam was catched/marked
i am now using
clam and Sanesecurity to eleminate pdf spam.

I have tried that, but clamav did not pick up one when scanning a
bunch of the pdf-spam.  I have used one of the downloadscripts on
Sanesecurity.  Do I have to do some other configurations to activate
the databasis for Clamav?

I have this in /var/lib/clamav at the moment:

  drwxr-xr-x  2 clamav clamav    4096 2007-07-12 14:22 
clamav-29a2fe02977a1d4c26abf3fd199d1e70
  -rw-r--r--  1 clamav clamav  995915 2007-07-11 22:48 daily.cvd
  -rwxrwxr--  1 clamav clamav       0 2007-07-12 14:15 .dbLock
  -rw-r--r--  1 clamav clamav 9351789 2007-07-11 22:48 main.cvd
  -rw-r--r--  1 clamav clamav  294979 2007-07-12 15:05 MSRBL-Images.hdb
  -rw-r--r--  1 clamav clamav  228436 2007-07-12 15:05 MSRBL-SPAM.ndb
  -rw-r--r--  1 clamav clamav  180868 2007-07-12 10:26 phish.ndb.gz
  -rw-r--r--  1 clamav clamav  115449 2007-07-12 10:26 scam.ndb.gz


The problem is with those last 2 files.

Those are the ones you're getting from Sanesecurity.  They're gzipped. 
In order to actually have ClamAV _USE_ them, you need to gunzip them.

This also make me wonder if you're actually testing the files before you 
put them into production.  If you're not, that's a rather bad idea.  At 
2am this morning, I had a non-usable phish.ndb come through.  If you're 
using clamd, that could have caused clamd to crash.


Here's the script I use for importing from MSRBL and Sanesecurity.  I 
run it out of cron with -all, on the hour.  You'll probably need to 
modify some bits of the first few lines (down to the rsync binary location):

#!/usr/local/bin/perl

my $chmod = "/bin/chmod";
my $mv = "/bin/mv";
my $gunzip = "/usr/bin/gunzip";
my $clamscan = "/usr/local/bin/clamscan";
my $testfile = "/bin/sh";
my $diff = "/usr/bin/diff";

my %methods =
   ("http"  => "/usr/local/bin/wget -q",
    "rsync" => "/usr/bin/rsync -q");

my %urls =
   ("msrbl-spam" => "rsync://rsync.mirror.msrbl.com/msrbl/MSRBL-SPAM.ndb",
    "msrbl-imgs" => 
"rsync://rsync.mirror.msrbl.com/msrbl/MSRBL-Images.hdb",
    "sane-phish" => 
"http://www.sanesecurity.com/clamav/phishsigs/phish.ndb.gz";,
    "sane-scam"  => 
"http://www.sanesecurity.com/clamav/scamsigs/scam.ndb.gz";);

my %tmpdirs =
   ("msrbl-spam" => "/tmp/msrbl",
    "msrbl-imgs" => "/tmp/msrbl",
    "sane-phish" => "/tmp/sanecomputing",
    "sane-scam"  => "/tmp/sanecomputing");


my %destdirs =
   ("msrbl-spam" => "/var/lib/clamav",
    "msrbl-imgs" => "/var/lib/clamav",
    "sane-phish" => "/var/lib/clamav",
    "sane-scam"  => "/var/lib/clamav");


my $getall = 0;
my (@distros, $dist, $tmpdir, $proto, $method, $file, $retcode);
my ($ufile, $diffout, $destdir);

if ($ARGV[0] =~ "--?al?l?") {
   $getall = 1;
   @distros = keys(%urls);
   }
else {
   @distros = @ARGV;
   }

foreach $dist (sort (@distros)) {
   $tmpdir = $tmpdirs{$dist};
   $destdir = $destdirs{$dist};
   $url = $urls{$dist};
   $proto = $url; $proto =~ s/:.*$//;
   $method = $methods{$proto};
   $file = $url; $file =~ s"^.*/([^/]*)$"$1";
   $ufile = $file; $ufile =~ s/\.gz$//;

   if ((-e $tmpdir) && (!(-d $tmpdir))) {
      rename ($tmpdir, ($tmpdir . ".bad"))
         || die "tmpdir $tmpdir isn't a directory, can't rename it";
      mkdir ($tmpdir) || die "can't make tmpdir $tmpdir";
      }
   elsif (! (-e $tmpdir)) {
      mkdir ($tmpdir) || die "can't make tmpdir $tmpdir";
      }
   system ("$chmod 700 $tmpdir");

   if ((-e $destdir) && (!(-d $destdir))) {
      rename ($destdir, ($destdir . ".bad"))
         || die "destdir $destdir isn't a directory, can't rename it";
      mkdir ($destdir) || die "can't make destdir $destdir";
      }
   elsif (! (-e $destdir)) {
      mkdir ($destdir) || die "can't make destdir $tmpdir";
      }
   system ("$chmod 775 $destdir");

   chdir ($tmpdir);

   if (-e $file) {
      unlink ($file);
      }

   if (-e $ufile) {
      unlink ($ufile);
      }

   # download the file
   if ($proto eq "rsync") {
      system("$method $url $file");
      }
   elsif ($proto eq "http") {
      system("$method $url");
      }

   unless (-e $file) {
      print "   didn't get download file $file\n";
      last;
      }

   if ($file =~ /\.gz$/) {
      system("$gunzip $file");
      $file = $ufile;
      }

   # test against clamav
   $retcode =
      system("$clamscan --database=$tmpdir $testfile > /dev/null 2>&1") 
/ 256;

   if ($retcode == 0) {
      # clamscan of testfile worked and didn't find a virus
      # lets see if it's the same file we already have/had
      $diffout = (system("$diff --brief --speed-large-files 
$tmpdir/$file $destdir/$file > /dev/null 2>/dev/null")) / 256;
      if ($diffout == 0) {
         # file hasn't changed
         unlink ($file);
         }
      else {
         print "   $file appears to have changed, moving to destination\n";
         system("$mv $tmpdir/$file $destdir/$file");
         system("$chmod 644 $destdir/$file");
         }
      }
   elsif ($retcode == 1) {
      print "   found a virus in $testfile while testing $dist\n";
      }
   else {
      print "   new $dist download appears to be corrupt\n";
      }
   }