Wietse Venema put forth on 1/30/2010 7:14 PM:
> Stan Hoeppner:
>> AFAIK I don't use Berkeley DB tables, only hash (small,few) and cidr
>> (very large, a handful).
>
> hash (and btree) == Berkeley DB.
Ahh, good to know. I'd thought only btree used Berkeley DB and that hash tables
used something else.
> If you have big CIDR tables, you can save lots of memory by using
> proxy:cidr: instead of cidr: (and running "postfix reload").
> Effectively, this turns all that private memory into something that
> can be shared via the proxy: protocol.
I implemented proxymap but it doesn't appear to have changed the memory
footprint of smtpd much at all, if any. I reloaded once, and restarted once
just in case.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4554 postfix 20 0 20828 17m 2268 S 0 4.5 0:00.46 smtpd
4560 postfix 20 0 20036 16m 2268 S 0 4.3 0:00.47 smtpd
4555 postfix 20 0 6812 3056 1416 S 0 0.8 0:00.10 proxymap
> The current CIDR implementation is optimized to make it easy to
> verify for correctness, and is optimized for speed when used with
> limited lists of netblocks (mynetworks, unassigned address blocks,
> reserved address blocks, etc.).
Understood.
> If you want to list large portions of Internet address space such
> as entire countries the current implementation starts burning CPU
> time (it examines all CIDR patterns in order; with a bit of extra
> up-front work during initialization, address lookups could skip
> over a lot of patterns, but the implementation would of course be
> harder to verify for correctness), and it wastes 24 bytes per CIDR
> rule when Postfix is compiled with IPv6 support (this roughly
> doubles the amount memory that is used by CIDR tables).
I don't really notice much CPU burn on any postfix processes with these largish
CIDRs, never have. I've got 12,212 CIDRs in 3 files, 11,148 of them in just the
"countries" file alone. After implementing proxymap, I'm not seeing much
reduction in smtpd RES size, maybe 1MB if that. SHR is almost identical to
before. If it's not the big tables bloating smtpd, I wonder what is? Or, have
I not implemented proxymap correctly? Following are my postconf -n and main.cf
relevant parts.
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
disable_vrfy_command = yes
header_checks = pcre:/etc/postfix/header_checks
inet_interfaces = all
message_size_limit = 10240000
mime_header_checks = pcre:/etc/postfix/mime_header_checks
mydestination = hardwarefreak.com
myhostname = greer.hardwarefreak.com
mynetworks = 192.168.100.0/24
myorigin = hardwarefreak.com
parent_domain_matches_subdomains = debug_peer_list smtpd_access_maps
proxy_interfaces = 65.41.216.221
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
$virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
$relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
$recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
$sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs
readme_directory = /usr/share/doc/postfix
recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
relay_domains =
smtpd_banner = $myhostname ESMTP Postfix
smtpd_helo_required = yes
smtpd_recipient_restrictions = permit_mynetworks
reject_unauth_destination check_recipient_access
hash:/etc/postfix/whitelist check_sender_access hash:/etc/postfix/whitelist
check_client_access hash:/etc/postfix/whitelist check_client_access
hash:/etc/postfix/blacklist check_client_access
regexp:/etc/postfix/fqrdns.regexp check_client_access
pcre:/etc/postfix/ptr-tld.pcre check_client_access proxy:${cidr}/countries
check_client_access proxy:${cidr}/spammer check_client_access
proxy:${cidr}/misc-spam-srcs reject_unknown_reverse_client_hostname
reject_non_fqdn_sender reject_non_fqdn_helo_hostname
reject_invalid_helo_hostname reject_unknown_helo_hostname
reject_unlisted_recipient reject_rbl_client zen.spamhaus.org
check_policy_service inet:127.0.0.1:60000
strict_rfc821_envelopes = yes
virtual_alias_maps = hash:/etc/postfix/virtual
/etc/postfix/main.cf snippet
cidr=cidr:/etc/postfix/cidr_files
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
$virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
$relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
$recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
$sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs
check_client_access proxy:${cidr}/countries
check_client_access proxy:${cidr}/spammer
check_client_access proxy:${cidr}/misc-spam-srcs
--
Stan
