Re: NOTE: Warning to Abusers of Update Servers

2017-11-25 Thread David Jones
On 11/25/2017 06:35 AM, Rupert Gallagher wrote: I hardly see any difference: with or without sa updates, the sa grades are always too low to be useful. If I disable my own filters, and let sa do the job with its default rules, the ham/spam ratio is ~10%-90%. With my rules, I get ~95%-5% inste

Re: NOTE: Warning to Abusers of Update Servers

2017-11-25 Thread Rupert Gallagher
I hardly see any difference: with or without sa updates, the sa grades are always too low to be useful. If I disable my own filters, and let sa do the job with its default rules, the ham/spam ratio is ~10%-90%. With my rules, I get ~95%-5% instead. I think that sa is not different from other re

Re: NOTE: Warning to Abusers of Update Servers

2017-11-25 Thread Jens Schleusener
On Thu, 23 Nov 2017, Kevin A. McGrail wrote: On 11/23/2017 6:31 PM, Dave Warren wrote: Would more mirrors be useful? I've got a ton of spare upstream bandwidth and am in the progress of setting up a few mirrors for other projects. Sure.  Always helps to spread the load more. All you have to

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread Dave Warren
Alright, it might be live at http://sa-update.razx.cloud/ Currently I don't do any logging of mirror traffic, although this may change in the near future. On Fri, Nov 24, 2017, at 05:02, Kevin A. McGrail wrote: > I really don't pay too much attention to bandwidth and you will want > to use http.

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread Dave Warren
On Fri, Nov 24, 2017, at 09:45, RW wrote: > On Fri, 24 Nov 2017 08:23:21 -0700 > Dave wrote: > > >> It mostly shouldn't, but when I was supporting a mail server that > > >> included a SpamAssassin integration, we ran into a non-zero number > > >> of installations where DNS checks failed and they f

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread RW
On Fri, 24 Nov 2017 08:23:21 -0700 Dave wrote: > >> It mostly shouldn't, but when I was supporting a mail server that > >> included a SpamAssassin integration, we ran into a non-zero number > >> of installations where DNS checks failed and they fell back on > >> direct connections. > > > > I do

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread Dave
My recollection is that something was eating the TXT results; but not the A records. Probably a PIX or something like that, it broke ESMTP pretty badly too. > On Nov 24, 2017, at 06:34, RW wrote: > > On Thu, 23 Nov 2017 16:39:25 -0700 > Dave Warren wrote: > >>> On 2017-11-21 11:57, RW wrot

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread David Jones
On 11/23/2017 09:08 PM, Dave Warren wrote: On Thu, Nov 23, 2017, at 16:01, Kevin A. McGrail wrote: On 11/23/2017 6:31 PM, Dave Warren wrote: Would more mirrors be useful? I've got a ton of spare upstream bandwidth and am in the progress of setting up a few mirrors for other projects. Sure.  A

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread RW
On Thu, 23 Nov 2017 16:39:25 -0700 Dave Warren wrote: > On 2017-11-21 11:57, RW wrote: > > On Tue, 21 Nov 2017 08:55:34 -0600 > > David Jones wrote: > > > > > >> You are correct. I haven't dug into the code to verify but it > >> appears that 3.4.x sa-update does use the DNS TXT record to know

Re: NOTE: Warning to Abusers of Update Servers

2017-11-24 Thread Kevin A. McGrail
I really don't pay too much attention to bandwidth and you will want to use http. We typically set new mirrors at the weight of 1 and then you can let us know if we can bump it up. Regards, KAM On November 23, 2017 10:08:06 PM EST, Dave Warren wrote: >On Thu, Nov 23, 2017, at 16:01, Kevin A. M

Re: NOTE: Warning to Abusers of Update Servers

2017-11-23 Thread Dave Warren
On Thu, Nov 23, 2017, at 16:01, Kevin A. McGrail wrote: > On 11/23/2017 6:31 PM, Dave Warren wrote: > > Would more mirrors be useful? I've got a ton of spare upstream > > bandwidth and am in the progress of setting up a few mirrors for other > > projects. > > > Sure.  Always helps to spread the l

Re: NOTE: Warning to Abusers of Update Servers

2017-11-23 Thread Kevin A. McGrail
On 11/23/2017 6:31 PM, Dave Warren wrote: Would more mirrors be useful? I've got a ton of spare upstream bandwidth and am in the progress of setting up a few mirrors for other projects. Sure.  Always helps to spread the load more. All you have to do is setup sa-update.XYZ.tld and add an rsyn

Re: NOTE: Warning to Abusers of Update Servers

2017-11-23 Thread Dave Warren
On 2017-11-21 11:57, RW wrote: On Tue, 21 Nov 2017 08:55:34 -0600 David Jones wrote: You are correct. I haven't dug into the code to verify but it appears that 3.4.x sa-update does use the DNS TXT record to know when to download so it doesn't hurt anything to run this version hourly. By th

Re: NOTE: Warning to Abusers of Update Servers

2017-11-23 Thread Dave Warren
Would more mirrors be useful? I've got a ton of spare upstream bandwidth and am in the progress of setting up a few mirrors for other projects. On 2017-11-21 10:47, Kevin A. McGrail wrote: My goal is to stop abuse without causing undue grief or fps. It may come to more draconian steps as you s

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread RW
On Tue, 21 Nov 2017 08:55:34 -0600 David Jones wrote: > You are correct. I haven't dug into the code to verify but it > appears that 3.4.x sa-update does use the DNS TXT record to know when > to download so it doesn't hurt anything to run this version hourly. By the sound of it this warning do

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
My goal is to stop abuse without causing undue grief or fps. It may come to more draconian steps as you suggest. Regards, KAM On November 21, 2017 10:13:38 AM EST, AJ Weber wrote: > >> The major offenders are sa-update 3.3.x and generic curl clients >based >> on the user agent in the logs runn

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread AJ Weber
The major offenders are sa-update 3.3.x and generic curl clients based on the user agent in the logs running from every minute to every 15 minutes and blindly pulling down the same rulesets over and over. My "vote" counts for very, very little, but since these clients already have the latest

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread David Jones
On 11/21/2017 03:56 AM, A. Schulze wrote: Kevin A. McGrail: If you are checking the SpamAssassin updates more than 2x a day, expect to be blocked in the very near future.  We have people checking literally every minute and we only release rules currently 1x per day.  There is no need to chec

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Eray Aslan
On Tue, Nov 21, 2017 at 01:41:56PM +0100, Ralf Hildebrandt wrote: > Yup, seen that on a stratum 1 NTP server - all clients seem to have > the same config, thus querying my server by means of NAT instead of > using an internal stratum 2 :( Common, probably internal, recursive DNS server but not an

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
On 11/21/2017 7:32 AM, Anthony Cartmell wrote: would give admins enough of a clue to see if they were a culprit without giving the actual IPs away? It's a good idea.  I think removing the first octet would be enough obfuscation. So these IPs in 21 days are the top 15 abusers.  With rule upda

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Ralf Hildebrandt
* Kevin A. McGrail : > On 11/21/2017 7:35 AM, Reindl Harald wrote: > > or they have simply more than one machine behind a single outgoing IP > Yes, we considered that too hence why we haven't just blocked things > outright. Yup, seen that on a stratum 1 NTP server - all clients seem to have the sa

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
On 11/21/2017 7:35 AM, Reindl Harald wrote: or they have simply more than one machine behind a single outgoing IP Yes, we considered that too hence why we haven't just blocked things outright. Regards, KAM

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Ralf Hildebrandt
* Kevin A. McGrail : > The Top 14 abusers account for nearly 30% of our update based on sampling > one mirror for people downloading the files. On my sanesecurity pattern mirror, I just blacklist/nullroute those idiots. -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ra

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Anthony Cartmell
>> I don't suppose you can list the offending IP addresses? >> >> Or partially-obfuscated IP addresses? > > While I think it might be helpful to the administrators, I thought it > might cross the line into publicly shaming people so I can't. Understood. Although perhaps listing them with just th

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
On 11/21/2017 7:11 AM, Anthony Cartmell wrote: I don't suppose you can list the offending IP addresses? Or partially-obfuscated IP addresses? While I think it might be helpful to the administrators, I thought it might cross the line into publicly shaming people so I can't. Regards, KAM

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Anthony Cartmell
> The Top 14 abusers account for nearly 30% of our update based on > sampling one mirror for people downloading the files. I don't suppose you can list the offending IP addresses? Or partially-obfuscated IP addresses? Anthony -- www.fonant.com - Quality web sites Tel. 01903 867 810 Fonant Ltd i

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
On 11/21/2017 4:56 AM, A. Schulze wrote: I use sa-update to update all channels I use. From what I've seen on "sa-update --debug" the default is to test an dns-record. ( "dig 1.4.3.updates.spamassassin.org. txt" for the main channel ) If the DNS answer indicate the same version no HTTP requests

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Kevin A. McGrail
On 11/21/2017 4:42 AM, Matthew Broadhead wrote: I have a cron to check once per day.  Is that ok or do you think once per week is enough? Hi Matt, Once per day would be appropriate.  In fact, once an hour would be fine.  The check should just hit DNS. The Top 14 abusers account for nearly 30%

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread A. Schulze
Kevin A. McGrail: If you are checking the SpamAssassin updates more than 2x a day, expect to be blocked in the very near future.  We have people checking literally every minute and we only release rules currently 1x per day.  There is no need to check this often! I use sa-update to updat

Re: NOTE: Warning to Abusers of Update Servers

2017-11-21 Thread Matthew Broadhead
I have a cron to check once per day.  Is that ok or do you think once per week is enough? On 21/11/2017 03:04, Kevin A. McGrail wrote: All, If you are checking the SpamAssassin updates more than 2x a day, expect to be blocked in the very near future.  We have people checking literally every