Things I'm putting into the new Botnet version (which will be 0.5):
1) someone noticed that some MTA's (specifically CommuniGate Pro) don't
put the relay's RDNS into the Received headers, and thus Botnet 0.4
always triggered "NORDNS" when run on that MTA. In the new version, if
Botnet finds that the relay it's going to look at has no rdns in the
pseudo-header, then the _first_ time it looks it will try to lookup the
relay (and store it in the pseudo-header if it finds it; or store -1 if
not). From then on, it will give the right answer for the other Botnet
rules. This avoids the performance problem of "every Botnet rule does 1
or 2 DNS checks" that I tried to solve 1 or 2 versions ago, but does
mean that at least 1 DNS check will be done (by the first Botnet rule
that happens to get called) if the relay doesn't have RDNS. This might
happen even if you have network checks turned off. If you're concerned
about the small performance hit on this, then it might be a good idea to
run a caching name server on the host where Botnet runs.
(I had also considered only doing this if the user set a new config
option, "botnet_lame_mta_rdns", to 1 ... but I thought I'd try this first)
2) As suggested, I've added "botnet_pass_domains" -- regular
expressions, anchored to the end of the hostname string, that look for
domains to exempt from Botnet checks.
3) I modifed the "IP in hostname" check slightly. It used to look for
mixed deximal and hexidecimal octets in the hostname. This caused a
small problem with the following Received header:
Received: from badger07006.apple.com (badger07006.apple.com [17.254.6.173])
("ad" is hexadecimal for "173", and you can see "006" right in there,
therefore 2 octets are present in the hostname)
To avoid this special case, I have made it so that it doesn't put the
hexicecimal and decimal checks into the same regular expression. This
could, however, slightly reduce Botnet's effectiveness. I'm going to
re-evaluate it over time.
(note: I have ALSO addressed this by putting apple\.com into the
botnet_pass_domains example; using botnet_pass_domains or botnet_pass_ip
might be the better way to address these special cases in the future,
but I'm not sure yet)
4) I've added "mx" to the included botnet_serverwords. Technically this
alone would exempt the ebay hosts that use "mxpool", so ebay wouldn't
need a botnet_skip_domains entry ... but I also made such an entry for
ebay. I'm not sure yet if "mx" is a good idea to have in
botnet_serverwords though.
5) In the past, I only had the 127.* localhost IP address block, and the
10.* private IP address block in the example botnet_skip_ip config.
From a suggestion I received, I've added the other two private IP
blocks as well ( 192.168.* and 172.(16-31).* ).
I have two questions:
Question 1: Someone suggested that, for botnet_pass_domains, I not
re-invent the wheel. SA already has several whitelist options
(whitelist* and sare_whitelist* were specifically mentioned). They
suggested that I leverage them. My first (two part) question is:
a) do any of them have a small enough value that they wouldn't counter
botnet's default score of 5? Meaning, if I "do nothing" with respect to
those other whitelist mechanisms, they'll still "do the right thing" and
let the botnet hosts through, right?
b) clearly I've gone ahead and done botnet_pass_domains ... but part of
me wants to "do both". So what is the right way to have Botnet
recognize those other host/domain whitelisting mechanisms?
I have no idea what the sare_whitelist entries look like, but I was
thinking maybe I could do take the whitelist_from argument, the 2nd
argument to whitelist_from_rcvd, and maybe the whitelist_from_spf
argument, and munge them into a domain name to exempt. The catch is: if
I do that, shouldn't I _also_ recognize the unwhitelist_* configs? That
starts to get a bit hairy, IMO.
For now, I'm not going to go down this path... but I'm interested in
people's opinions about whether or not I should recognize whitelist*,
sare_whitelist*, and unwhitelist* config options and somehow incorporate
them into botnet_pass_domains. I'd also consider code snippets that
would be compatible with the code I already have for
Botnet::parse_config. My main hope, though, is that the scores for
those mechanisms are already negative enough that they over-ride Botnet
anyway. Given that the ones in the base SA are scored at -6, -15, or
-100 ... I think that's a comfortable assumption on my part. I don't
know if sare_whitelist fits into that or not, though.
(for similar reasons I'm currently not going to look at making the
BOTNET meta rule's expression more complicated with references to DK and
DKIM; the DK scores in the base SA are scored at -100 and -7.5 ... that
seems useful enough to me; but I might look at putting in alternate meta
rule expressions that are commented out, if people really want me to;
that way people could just choose to comment and uncomment whatever
seems most appropriate for their situation)
Question 2: someone asked why my module is "Botnet" instead of
"Mail::SpamAssassin::Plugin::Botnet". The answer is: when I first
started this (and this is/was my first SA Plugin authoring attempt), I
tried that and it didn't work. If someone wants to look at it, and
figure out how to make that work (but still have the files located in
/etc/mail/spamassassin) I would happily incorporate it.
Last, someone offered to host this if I needed to. I appreciate the
offer. I may decide to bite the bullet and host this on sourceforge at
some point (assuming, say, the SA team doesn't like it enough to include
it in their standard examples) ... but for now my existing location is
working fine.
I expect to release the new version over the coming weekend.
