Stan Hoeppner:
[ Charset ISO-8859-1 unsupported, converting... ]
> Wietse Venema put forth on 11/23/2010 6:57 PM:
> > Victor Duchovni:
> >> On Tue, Nov 23, 2010 at 07:35:49PM -0500, Wietse Venema wrote:
> >>
> >>> Victor Duchovni:
> >>>> On Tue, Nov 23, 2010 at 05:20:11PM -0600, Stan Hoeppner wrote:
> >>>>
> >>>>> Would you please give us the run down on why these map types (and maybe
> >>>>> others) shouldn't be used with proxymap due to performance reasons? You
> >>>>> mentioned something about this long ago but I can't seem to locate that
> >>>>> email in my archives. IIRC you didn't go into much technical detail as
> >>>>> to why the performance would be lower using proxymap.
> >>>>
> >>>> There is no point in using IPC to ask a server to read a file, when the
> >>>> client process can read it directly. Especially with CDB, since there
> >>>> is no per-client page pool adding some per-client memory overhead.
> >>>
> >>> With Stan's huge CIDR maps on a small machine, proxymap helps
> >>> to avoid running out of memory.
> >>
> >> CIDR maps are indeed a different kettle of fish, they are not
> >> "read-a-file via IPC", rather they are "do a memory lookup via IPC", and
> >> if the memory footprint is sufficiently high (lots Berkeley DB tables or
> >> huge CIDR tables), then indeed one may want to use proxymap.
> >>
> >> With "cdb" (the recommended read-only indexed table type for Postfix)
> >> direct
> >> access is best.
> >
> > proxymap is OK for moderate traffic to expensive or otherwise
> > inaccessible resources. Opening a CDB map is not expensive, an
> > SQL database handle or memory-based map (regexp etc.) can be
> > expensive, and /etc/passwd can be inaccessible due to lazy opens.
> >
> > proxymap is not the best solution if you need minimal latency at
> > all cost.
> >
> > Wietse
>
> Thank you gentlemen. I relatively recently added the dnswl.org CIDR
> whitelist table, which contains over 97K+ entries. I use a perl script
> to consolidate the CIDRs, cutting it down to around 67K entries. Timing
> postmap -q it takes on average .5 seconds to complete a lookup in the
> consolidated file regardless of where the entry lies within the file.
> Timing queries on the original file shows .75 seconds per lookup. This
> is after caching. Testing cold is kind of irrelevant. I'm guessing the
> latency is actually higher when smtpd queries proxymap than when timing
> postmap -q. Is this the case?
That's 0.5 seconds to read the table once, and milliseconds to query it.
Wietse
> An extra ~.5-1.0 seconds isn't a big deal regarding this MX. However,
> each new table or dns lookup I add to my anti-spam checks adds up, and I
> don't want to reach a point where I'm taking 10 seconds per connection
> to determine if I'm going to reject it. So I try to continually
> optimize, especially after adding something like the dnswl.org CIDR map.
>
> --
> Stan
>
>
