Sorry for top posting. Forgot to add something earlier: Proxymap seems to be
exiting on my system immediately after servicing requests. It does not seem to
be obeying $max_use or $max_idle which are both set to 100. It did this even
before I added cidr lists to proxymap a few hours ago. Before that, afaik, it
was only being called for local alias verification, and it exited immediately in
that case as well.
--
Stan
Stan Hoeppner put forth on 1/30/2010 11:13 PM:
> Wietse Venema put forth on 1/30/2010 7:14 PM:
>> Stan Hoeppner:
>>> AFAIK I don't use Berkeley DB tables, only hash (small,few) and cidr
>>> (very large, a handful).
>>
>> hash (and btree) == Berkeley DB.
>
> Ahh, good to know. I'd thought only btree used Berkeley DB and that hash
> tables
> used something else.
>
>> If you have big CIDR tables, you can save lots of memory by using
>> proxy:cidr: instead of cidr: (and running "postfix reload").
>> Effectively, this turns all that private memory into something that
>> can be shared via the proxy: protocol.
>
> I implemented proxymap but it doesn't appear to have changed the memory
> footprint of smtpd much at all, if any. I reloaded once, and restarted once
> just in case.
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 4554 postfix 20 0 20828 17m 2268 S 0 4.5 0:00.46 smtpd
> 4560 postfix 20 0 20036 16m 2268 S 0 4.3 0:00.47 smtpd
> 4555 postfix 20 0 6812 3056 1416 S 0 0.8 0:00.10 proxymap
>
>> The current CIDR implementation is optimized to make it easy to
>> verify for correctness, and is optimized for speed when used with
>> limited lists of netblocks (mynetworks, unassigned address blocks,
>> reserved address blocks, etc.).
>
> Understood.
>
>> If you want to list large portions of Internet address space such
>> as entire countries the current implementation starts burning CPU
>> time (it examines all CIDR patterns in order; with a bit of extra
>> up-front work during initialization, address lookups could skip
>> over a lot of patterns, but the implementation would of course be
>> harder to verify for correctness), and it wastes 24 bytes per CIDR
>> rule when Postfix is compiled with IPv6 support (this roughly
>> doubles the amount memory that is used by CIDR tables).
>
> I don't really notice much CPU burn on any postfix processes with these
> largish
> CIDRs, never have. I've got 12,212 CIDRs in 3 files, 11,148 of them in just
> the
> "countries" file alone. After implementing proxymap, I'm not seeing much
> reduction in smtpd RES size, maybe 1MB if that. SHR is almost identical to
> before. If it's not the big tables bloating smtpd, I wonder what is? Or,
> have
> I not implemented proxymap correctly? Following are my postconf -n and
> main.cf
> relevant parts.
>
> alias_maps = hash:/etc/aliases
> append_dot_mydomain = no
> biff = no
> config_directory = /etc/postfix
> disable_vrfy_command = yes
> header_checks = pcre:/etc/postfix/header_checks
> inet_interfaces = all
> message_size_limit = 10240000
> mime_header_checks = pcre:/etc/postfix/mime_header_checks
> mydestination = hardwarefreak.com
> myhostname = greer.hardwarefreak.com
> mynetworks = 192.168.100.0/24
> myorigin = hardwarefreak.com
> parent_domain_matches_subdomains = debug_peer_list smtpd_access_maps
> proxy_interfaces = 65.41.216.221
> proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
> $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
> $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
> $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
> $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
> proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs
> readme_directory = /usr/share/doc/postfix
> recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
> relay_domains =
> smtpd_banner = $myhostname ESMTP Postfix
> smtpd_helo_required = yes
> smtpd_recipient_restrictions = permit_mynetworks
> reject_unauth_destination check_recipient_access
> hash:/etc/postfix/whitelist check_sender_access hash:/etc/postfix/whitelist
> check_client_access hash:/etc/postfix/whitelist check_client_access
> hash:/etc/postfix/blacklist check_client_access
> regexp:/etc/postfix/fqrdns.regexp check_client_access
> pcre:/etc/postfix/ptr-tld.pcre check_client_access proxy:${cidr}/countries
> check_client_access proxy:${cidr}/spammer check_client_access
> proxy:${cidr}/misc-spam-srcs reject_unknown_reverse_client_hostname
> reject_non_fqdn_sender reject_non_fqdn_helo_hostname
> reject_invalid_helo_hostname reject_unknown_helo_hostname
> reject_unlisted_recipient reject_rbl_client zen.spamhaus.org
> check_policy_service inet:127.0.0.1:60000
> strict_rfc821_envelopes = yes
> virtual_alias_maps = hash:/etc/postfix/virtual
>
> /etc/postfix/main.cf snippet
>
> cidr=cidr:/etc/postfix/cidr_files
>
> proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
> $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
> $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
> $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
> $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
> proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs
>
> check_client_access proxy:${cidr}/countries
> check_client_access proxy:${cidr}/spammer
> check_client_access proxy:${cidr}/misc-spam-srcs
>
