On Tue, 18 Nov 2003, Larry Gilson wrote: > Most of you guys can get over my head quickly.
Read, learn, and exceed us. :-) > This sounds a lot like the squidGuard blacklist implementation. You start > with a base text file - one each for domains, urls, and regex. It is up to > each the administrator to convert to db at each installation. At issue here is a database that'll be 99.9% read-only; the only modifications to the DB structure would be the (inevitable) additions and deletions. Having something the size of a B-tree "evilrules" database would, I suspect, severly tax less-capable iron and _really_ tax heavily-hit spam-scanners. > So if you want to share site specific data, you can supply the > .diff file. What would be beneficial here though would be to create the > update mechanism so it will update any file#.diff entry (in the same manner > that SA will parse any .cf file for rules) while ignoring existing file.db > entries. I actually think that the best approach to this, other than creating a truly dynamic net-based system (which would get the DDOS of death pretty quickly) would be to simply dump the DB file to plain-text, make a "diff" of your local site and whatever other site you're interested in using, and add in (or subtract) the diffs (remembering that in DBs the _writes_ are what hurts). > The porn domains list alone has 48536 entries. My code builds this list > fairly slowly using DB_File. BerkeleyDB, which can not be used with > squidGuard, would be similar but only slightly faster. Full writes of the entire DB should be fairly uncommon, and even then might be handled "off-line" to the operational SA engine. Limited updates ought to be the order of the day rather than wholesale rewrites. > Regardless, the squidGuard authors have mentioned in the documentation > that using pre-built databases performs marginally slower even with a > 1 million entry database compared to the in-memory-only B-trees. > > References: > http://www.squidguard.org/doc/ > News and changes --> Major news and changes in --> 1.1.0.beta1: This might be "required reading" for those interested in integrating this sort of technology into SA. I'll take a browse through it in the morning. Cheers. +------------------------------------------------+---------------------+ | Carl Richard Friend (UNIX Sysadmin) | West Boylston | | Minicomputer Collector / Enthusiast | Massachusetts, USA | | mailto:[EMAIL PROTECTED] +---------------------+ | http://users.rcn.com/crfriend/museum | ICBM: 42:22N 71:47W | +------------------------------------------------+---------------------+ ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. Does SourceForge.net help you be more productive? Does it help you create better code? SHARE THE LOVE, and help us help YOU! Click Here: http://sourceforge.net/donate/ _______________________________________________ Spamassassin-talk mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/spamassassin-talk
