Hi!
I just threw together a plugin that can check URLs for redirections, and
add whatever they redirect to to the message meta-data so that the true
destinations are checked by URIBLs etc.
It doesn't do this for all URLs in a message. I will only follow those
URLs it is specifically told to follow. Also, it only asks for HEAD
rather than pages in order to keep the traffic down.
I'm not sure wether this is really worthwhile or if it is just a waste
of time and resources, but the idea is to use it for URL shorteners that
are beeing abused by spammers.
To be really useful it needs a list of abused URL shorteners. I don't
know wich shorteners are most abused, so I don't know what the list
should contain.
(The three example shorteners are in the POD because I knew about them,
not because they are beeing used by spammers.)
If anyone thinks this is a good idea you can check the plugin at
<http://whatever.frukt.org/spamassassin.text.shtml?accessibler#URLRedirect.pm>.
Suggestions and criticism are very welcome.
URL shortener addresses (with formats) even more welcome.
Notes:
This is not extensively tested. It may well contain bugs. It's not a
finished thing.
If this plugin is a good idea, making it do it's HEAD requests in
paralell would be a good idea, but I don't know what the best way to do
that in perl for SA would be. (Currently it has a hardcoded timeout of
10 seconds around it's requesting stage, but no other time saving stuff.)
Using a cache should also be implemented so that repeatedly seen URLs
aren't followed over and over again. This should be pretty simple.
Since it needs URL meta-data to be checked before it runs, and needs to
add it's own meta-data before the rest of the scan run, it can't really
work asyncronoulsy AFAICS. Currently it uses a parsed_metadata at
priority -1 in order to add it's own meta-data. Maybe this isn't the
right way to do this.
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
