Tim
thanks for the response. I guess the issues I saw (and others on the list) were as a result of you using all your bandwidth down to the abusers.
Anyway ta for the ruleset. Maybe one day people will also update all their old MailScanner hosts (or configure then not to bounce) and you can take those rules out of the set ;-)
-- Martin Hepworth Snr Systems Administrator Solid State Logic Tel: +44 (0)1865 842300
Tim Jackson wrote:
Martin Hepworth <martinh <at> solid-state-logic.com> writes:
Tim does seem to have quite a few problems with people getting to this on a regular basis.
(Hello; I don't follow the SA list routinely at the moment purely due to volume & time pressures)
I'm not sure I'm aware of "regular" problems. To put some certainty to the speculation in this thread and various theories I've received, as of last week I implemented a rate limit whereby each unique IP address may only do one HTTP GET and one HTTP HEAD request on the ruleset once in any given 24 hour period. (Actually, thanks to a helpful suggestion by Matthew Turnbull, it's slightly less than 24 hours to take account of cronjobs not running at exactly the same time each day, but the principle is the same.).
I'm sorry if it has caused any inconvenience to anyone but I have had to do this to counter the abuse caused by a very small minority of users who eat my bandwidth downloading the (large) ruleset very regularly, often without even checking (via HTTP HEAD or conditional HTTP GET) whether it has changed. In the (probably unlikely) event that anyone reading this is one of those idiots who has a script that unconditionally downloads 100K off my site once a minute, 24 hours a day: stop being so downright selfish.
I appreciate that the "once a day per IP" is not a particularly good solution, particularly for those behind a transparent proxy or on a large NAT network but in the absence of any better ideas that's what I've done for now. If it is causing anyone particular problems for any reason, please do contact me and I will try to work something out. I'm not trying to stop anyone responsible having reasonable access to the list.
For everyone else: check once a day or less and you shouldn't even notice the restrictions as long as you have your own unique IP. If you're doing that and still having problems, let me know. There's absolutely no need to check more than once per day.
If you're in doubt about what's going on, visit the URL in a web browser from the machine you are trying to download the rules from (use lynx/elinks or whatever if you're on a command-line-administered server). You will get a clear error message telling you why you can't access it: if you're requesting the page too soon after your last request you will get a message similar to this:
"Sorry, someone (possibly you) using your IP address (XX.XX.XX.XX) performed a successful GET request on this page not long ago. Please do not check this page more frequently than once per day."
where XX.XX.XX.XX is the requesting IP seen from my end. This is a temporary, dynamic block which will automatically clear 24 hours after your last *successful* request (meaning that if, for example, you check every 6 hours, you will not keep resetting the timer and thus be refused forever; approximately 1 in 4 checks will succeed).
There are a small number of IPs that I have permanently blacklisted for more serious abuse. In the unlikely event that you're on that list, you will get a different page entitled "BANNED FOR ABUSIVE BEHAVIOUR" explaining that you need to contact me. In that case, you will not be able to access the list from the IP in question unless I manually remove you. I should note that this blacklist is not new; I have been blacklisting some IPs for a while.
Hope that clears it all up and again apologies for any inconvenience.
Tim
**********************************************************************
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager.
This footnote confirms that this email message has been swept for the presence of computer viruses and is believed to be clean.
**********************************************************************
