On Tue, Jan 26, 2010 at 04:06:29PM +0100, Erik Sonn wrote:
>
> Dear everyone,
>
> I'm working on some Antispam-Proxy, using Postfix as MTA. Postfix is
> 2.6.2-RC1 on an Ubuntu 8.04 LTS base-system.
>
>
> Preconditions:
> * Postfix shall only accept mails addressed to valid (=existing)
> recipients. To accomplish this, I'm using a regexp:/ map on
> relay_recipient_maps (the specific file is called "usermaps").
> * This usermaps file is automatically generated from an hourly cron-job,
> fetching all valid email-addresses via LDAP (however, the Postfix
> installation doesn't care about LDAP at all, this is autonomously done
> by some perl script).
> * The data gathered from LDAP is stuffed into a temporary file until
> finished, and then "atomatically" copied over the original usermaps
> file, before Postfix is triggered to reload.
>
> Problem:
> * At very irregular intervals, varying in time and quantity, Postfix
> refuses to accept Mails because the recipient address is seemingly
> unknown, altough that specific mail address (changes every time,
> unpredictable) is correctly defined in the usermaps file. The
> log-messages are like:
just curious, why regexp-ing and not dumping a valid postmap input
file for the relay_rcpt map: u...@domain OK ?
rebuilding the map with postmap will help with an exclusive lock on
the file so the readers wont get fooled by the update process.
if postmaps doesn't sound good, try "moving" instead "copying" the
regexp map. meaning: generate the ldap dump in a temporary file and
mv that to postfix regexp map file . that should eliminate surprises.
p.s. what about postfix direct ldap queries?
>
> 2010-01-26T15:10:29+01:00 hostmail postfix/smtpd[22884]: NOQUEUE:
> reject: RCPT from smtp.citrix.com[66.165.176.89]: 550 5.1.1
> <alexxxx...@xxxxxxx.de>: Recipient address rejected: User unknown in
> relay recipient table; from=<no.repl...@citrix.com>
> to=<alexxxxx...@xxxxxxxx.de> proto=ESMTP helo=<SMTP.CITRIX.COM>
>
> * Assuming the hourly cron-job is executed 24 times a day, 1-4 times
> Postfix logs the following message:
>
> 2010-01-26T08:57:25+01:00 hostmail postfix/smtpd[3398]: warning: regexp
> map /etc/postfix/usermaps, line 2434: no closing regexp delimiter "/":
> skipping this rule
>
> The lines-number is always randomly changing, and I have made quite some
> effort to make sure that the usermaps file is always complete,
> syntactically correct and consistent. As you see, the logentry above is
> timed "08:57:25" (the cron-job begins fetching addresses via LDAP always
> at *:57).
> Interestingly, my 'watch stat /etc/postfix/usermaps' shows this:
>
> # Before the 08:57 cron-job touches usermaps
> @Tue Jan 26 08:57:24 CET 2010
> Access: 2010-01-26 07:57:24.000000000 +0100
> Modify: 2010-01-26 07:57:22.000000000 +0100
> Change: 2010-01-26 07:57:22.000000000 +0100
>
> # After the 08:57 cron-job re-wrote usermaps, but Postfix hasn't read it
> # yet
> @Tue Jan 26 08:57:26 CET 2010
> Access: 2010-01-26 08:57:25.000000000 +0100
> Modify: 2010-01-26 08:57:25.000000000 +0100
> Change: 2010-01-26 08:57:25.000000000 +0100
>
> # After Postfix read the new usermaps after reloading
> @Tue Jan 26 08:57:36 CET 2010
> Access: 2010-01-26 08:57:35.000000000 +0100
> Modify: 2010-01-26 08:57:25.000000000 +0100
> Change: 2010-01-26 08:57:25.000000000 +0100
>
> If you look at these times, the file is *read* by Postfix at 08:57:35,
> but the log-line above claims the warning at 07:57:25. How can this be?
> The 10 seconds delay is because of an intended sleep() between writing
> the usermaps and reloading Postfix.
>
> Moreover, when mails a rejected as described above, the *time* these
> rejects happen do not seem to correlate with the regexp-warnings, nor do
> the rejected recipient mail-addresses. It seems like everything happens
> quite random here.
>
> What I've already checked:
> * Generation of usermaps file is OK and always succeeds. All addresses
> are successfully fetched, the file is writen syntactically correct and
> complete.
> * I/O- and buffering-issues have been tested and shouldn't be the
> problem (e.g. reloading Postfix while I/O buffer hasn't been flushed
> yet).
> * The basic Postfix configuration works perfectly and never made any
> troubles. That usermaps issue seems to occur only then the usermaps is
> getting large (>1k lines; in this specific case, it's about 10k lines
> large).
>
> The installation runs on a virtualized platform, using XEN. Postfinger
> output is attached. I should also mention that, for various reasons,
> it's not *easily* possible for me to simply upgrade the Postfix version.
>
>
> Thank you very much,
> Erik
> postfinger - postfix configuration on Tue Jan 26 15:18:25 CET 2010
> version: 1.30
>
> Warning: postfinger output may show private configuration information,
> such as ip addresses and/or domain names which you do not want to show
> to the public. If this is the case it is your responsibility to modify
> the output to hide this private information. [Remove this warning with
> the --nowarn option.]
>
> --System Parameters--
> mail_version = 2.6.2-RC1
> hostname = hostmail
> uname = Linux hostmail 2.6.24-24-server #1 SMP Tue Jun 30 21:03:25 UTC 2009
> i686 GNU/Linux
>
> --Packaging information--
> looks like this postfix comes from deb package: postfix-2.6.2~rc1-1
>
> --main.cf non-default parameters--
> alias_maps = hash:/etc/aliases
> anvil_rate_time_unit = 30m
> append_dot_mydomain = no
> biff = no
> bounce_queue_lifetime = 1h
> broken_sasl_auth_clients = yes
> content_filter = smtp-amavis:[127.0.0.1]:10024
> header_checks = regexp:/etc/postfix/header_checks
> local_recipient_maps = hash:/etc/postfix/local_rcpt_map
> mailbox_size_limit = 0
> mailbox_transport_maps = hash:/etc/postfix/mbox_transport
> maximal_queue_lifetime = 6h
> message_size_limit = 500000000
> mydestination = localhost, $myhostname
> myhostname = hostmail.XXXXXXXX.de
> mynetworks = 127.0.0.0/8
> queue_minfree = 1000000000
> recipient_delimiter = +
> relay_domains = hash:/etc/postfix/transport
> relay_recipient_maps = regexp:/etc/postfix/usermaps
> smtpd_banner = $myhostname ANTISPAM PROXY
> smtpd_client_connection_rate_limit = 200
> smtpd_client_restrictions = check_client_access
> cidr:/etc/postfix/amavis_bypass_internal_warn, check_client_access
> cidr:/etc/postfix/amavis_bypass_internal_filter, check_client_access
> cidr:/etc/postfix/amavis_bypass_filter_smtpcrypt, check_client_access
> cidr:/etc/postfix/amavis_bypass_filter, check_client_access
> cidr:/etc/postfix/amavis_bypass_accept, check_client_access
> cidr:/etc/postfix/amavis_bypass_internal_accept,
> smtpd_data_restrictions = reject_unauth_pipelining,
> smtpd_helo_required = yes
> smtpd_recipient_restrictions = check_client_access
> cidr:/etc/postfix/amavis_bypass_internal_accept, check_recipient_access
> regexp:/etc/postfix/filter-quarantine.regexp, check_policy_service
> inet:127.0.0.1:10040, permit_sasl_authenticated, permit_mynetworks,
> reject_unauth_destination, reject_non_fqdn_sender,
> reject_unknown_sender_domain, permit
> smtpd_restriction_classes = rc_greylisting
> smtpd_sasl_authenticated_header = yes
> smtpd_sender_restrictions = permit_sasl_authenticated, check_sender_access
> regexp:/etc/postfix/amavis_senderbypass_filter, permit_mynetworks, permit
> smtpd_timeout = 60
> transport_maps = hash:/etc/postfix/transport
> virtual_gid_maps = static:114
> virtual_mailbox_base = /var/quarantine
> virtual_mailbox_limit = 1000000000
> virtual_mailbox_maps = hash:/etc/postfix/virtual_mbox
> virtual_uid_maps = static:106
>
> --master.cf--
> 0.0.0.0:smtp inet n - - - 48 smtpd
> pickup fifo n - - 60 1 pickup
> -o content_filter=
> cleanup unix n - - - 0 cleanup
> qmgr fifo n - n 300 1 qmgr
> tlsmgr unix - - - 1000? 1 tlsmgr
> rewrite unix - - - - - trivial-rewrite
> bounce unix - - - - 0 bounce
> defer unix - - - - 0 bounce
> trace unix - - - - 0 bounce
> verify unix - - - - 1 verify
> flush unix n - - 1000? 0 flush
> proxymap unix - - n - - proxymap
> smtp unix - - - - - smtp
> relay unix - - - - - smtp
> -o fallback_relay=
> showq unix n - - - - showq
> error unix - - - - - error
> discard unix - - - - - discard
> local unix - n n - - local
> virtual unix - n n - - virtual
> lmtp unix - - - - - lmtp
> anvil unix - - - - 1 anvil
> scache unix - - - - 1 scache
> maildrop unix - n n - - pipe
> flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
> uucp unix - n n - - pipe
> flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail
> ($recipient)
> ifmail unix - n n - - pipe
> flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
> bsmtp unix - n n - - pipe
> flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender
> $recipient
> scalemail-backend unix - n n - 2 pipe
> flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
> ${nexthop} ${user} ${extension}
> mailman unix - n n - - pipe
> flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
> ${nexthop} ${user}
> smtp-amavis unix - - n - 16 smtp
> -o smtp_data_done_timeout=1200
> -o smtp_send_xforward_command=yes
> -o disable_dns_lookups=yes
> -o max_use=20
> 127.0.0.1:10025 inet n - n - - smtpd
> -o content_filter=
> -o smtpd_restriction_classes=
> -o smtpd_delay_reject=no
> -o smtpd_client_restrictions=permit_mynetworks,reject
> -o smtpd_helo_restrictions=
> -o smtpd_sender_restrictions=
> -o smtpd_recipient_restrictions=permit_mynetworks,reject
> -o smtpd_data_restrictions=reject_unauth_pipelining
> -o smtpd_end_of_data_restrictions=
> -o mynetworks=127.0.0.0/8
> -o smtpd_error_sleep_time=0
> -o smtpd_soft_error_limit=1001
> -o smtpd_hard_error_limit=1000
> -o smtpd_client_connection_count_limit=0
> -o smtpd_client_connection_rate_limit=0
> -o smtpd_milters=
> -o local_header_rewrite_clients=
> -o local_recipient_maps=
> -o relay_recipient_maps=
> -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
> smtpcrypt unix - n n - - pipe
> flags=Rq user=smtpcrypt argv=/usr/local/bin/smtpcrypt.pl
> sasl${sasl_method} ${client_address} ${sender} ${recipient}
> retry unix - - - - - error
>
> -- end of postfinger output --
--
adrian ilarion ciobanu
adria...@ciobanu.name
http://pub.mud.ro/~cia
+40 788 319 497
