On 09/03/2015 12:18 PM, Martin Gregorie wrote:
On Thu, 2015-09-03 at 11:15 +0700, Olivier Nicole wrote:
Oh well, I will give a look at URIDNSBL and see whether/how I can
change
it.
Implementing a simple lookup server using a hashtable of a B-tree can
be very good performance, even from a single-threaded local server.
Back in 2000 I had a similar problem with looking (mobile) phone
numbers containing up to 19 digits. I used a red-black binary tree
because these are self-balancing. My first attempt ran at 1,500
lookups/sec on a 150 MHz DEC AlphaServer with a tree size of 500,000
phone numbers. After I rewrote the memory allocation code, which was
VERY slow in DEC's Mach-based UNIX, I got it up to 25,000 lookups/sec
on the same hardware.
The one thing to watch if you go this way is that, if your server
starts up by building the storage structure from a flat file there are
two possible problems:
(a) depending on how efficient the library code is, start-up may
take longer than you expect if the storage structure is rebuilt
from scratch each time.
(b) beware of data ordering in the source file, especially if its
sorted for easy maintenance, because this can create a degenerate
structure whose performance degenerates into a linear scan. This
can happen if a hashtable's hashing algorithm generates too many
key collisions or if a binary tree doesn't use the self-balancing
red-black algorithm.
I had a quick scan to see if there are any OSS key-lookup servers that
could be easily used as a local RBL but didn't find any.
Please excuse my ignorance....
but wouldn't a key:value server like Redis do the trick?
It can't get much faster than that.. ok.. maybe memcached
Even rbldnsd could probably deal with it using the generic dataset
(within DNS spec limits)
