On Tue, 2010-01-26 at 10:33:41 -0600, adrian ilarion ciobanu wrote:
> On Tue, Jan 26, 2010 at 04:06:29PM +0100, Erik Sonn wrote:
> >
> > Dear everyone,
> >
> > I'm working on some Antispam-Proxy, using Postfix as MTA. Postfix is
> > 2.6.2-RC1 on an Ubuntu 8.04 LTS base-system.
> >
> >
> > Preconditions:
> > * Postfix shall only accept mails addressed to valid (=existing)
> > recipients. To accomplish this, I'm using a regexp:/ map on
> > relay_recipient_maps (the specific file is called "usermaps").
> > * This usermaps file is automatically generated from an hourly cron-job,
> > fetching all valid email-addresses via LDAP (however, the Postfix
> > installation doesn't care about LDAP at all, this is autonomously done
> > by some perl script).
> > * The data gathered from LDAP is stuffed into a temporary file until
> > finished, and then "atomatically" copied over the original usermaps
> > file, before Postfix is triggered to reload.
> >
> > Problem:
> > * At very irregular intervals, varying in time and quantity, Postfix
> > refuses to accept Mails because the recipient address is seemingly
> > unknown, altough that specific mail address (changes every time,
> > unpredictable) is correctly defined in the usermaps file. The
> > log-messages are like:
>
> just curious, why regexp-ing and not dumping a valid postmap input
> file for the relay_rcpt map: u...@domain OK ?
> rebuilding the map with postmap will help with an exclusive lock on
> the file so the readers wont get fooled by the update process.
>
> if postmaps doesn't sound good, try "moving" instead "copying" the
> regexp map. meaning: generate the ldap dump in a temporary file and
> mv that to postfix regexp map file . that should eliminate surprises.
>
> p.s. what about postfix direct ldap queries?
I'm used to utilize regexp:/ maps for enforcing bounce address tag
validation support. However, moving instead of copying really did the
trick, the problem never occured again.
Thank you very much!
Erik
> >
> > 2010-01-26T15:10:29+01:00 hostmail postfix/smtpd[22884]: NOQUEUE:
> > reject: RCPT from smtp.citrix.com[66.165.176.89]: 550 5.1.1
> > <alexxxx...@xxxxxxx.de>: Recipient address rejected: User unknown in
> > relay recipient table; from=<no.repl...@citrix.com>
> > to=<alexxxxx...@xxxxxxxx.de> proto=ESMTP helo=<SMTP.CITRIX.COM>
> >
> > * Assuming the hourly cron-job is executed 24 times a day, 1-4 times
> > Postfix logs the following message:
> >
> > 2010-01-26T08:57:25+01:00 hostmail postfix/smtpd[3398]: warning: regexp
> > map /etc/postfix/usermaps, line 2434: no closing regexp delimiter "/":
> > skipping this rule
> >
> > The lines-number is always randomly changing, and I have made quite some
> > effort to make sure that the usermaps file is always complete,
> > syntactically correct and consistent. As you see, the logentry above is
> > timed "08:57:25" (the cron-job begins fetching addresses via LDAP always
> > at *:57).
> > Interestingly, my 'watch stat /etc/postfix/usermaps' shows this:
> >
> > # Before the 08:57 cron-job touches usermaps
> > @Tue Jan 26 08:57:24 CET 2010
> > Access: 2010-01-26 07:57:24.000000000 +0100
> > Modify: 2010-01-26 07:57:22.000000000 +0100
> > Change: 2010-01-26 07:57:22.000000000 +0100
> >
> > # After the 08:57 cron-job re-wrote usermaps, but Postfix hasn't read it
> > # yet
> > @Tue Jan 26 08:57:26 CET 2010
> > Access: 2010-01-26 08:57:25.000000000 +0100
> > Modify: 2010-01-26 08:57:25.000000000 +0100
> > Change: 2010-01-26 08:57:25.000000000 +0100
> >
> > # After Postfix read the new usermaps after reloading
> > @Tue Jan 26 08:57:36 CET 2010
> > Access: 2010-01-26 08:57:35.000000000 +0100
> > Modify: 2010-01-26 08:57:25.000000000 +0100
> > Change: 2010-01-26 08:57:25.000000000 +0100
> >
> > If you look at these times, the file is *read* by Postfix at 08:57:35,
> > but the log-line above claims the warning at 07:57:25. How can this be?
> > The 10 seconds delay is because of an intended sleep() between writing
> > the usermaps and reloading Postfix.
> >
> > Moreover, when mails a rejected as described above, the *time* these
> > rejects happen do not seem to correlate with the regexp-warnings, nor do
> > the rejected recipient mail-addresses. It seems like everything happens
> > quite random here.
> >
> > What I've already checked:
> > * Generation of usermaps file is OK and always succeeds. All addresses
> > are successfully fetched, the file is writen syntactically correct and
> > complete.
> > * I/O- and buffering-issues have been tested and shouldn't be the
> > problem (e.g. reloading Postfix while I/O buffer hasn't been flushed
> > yet).
> > * The basic Postfix configuration works perfectly and never made any
> > troubles. That usermaps issue seems to occur only then the usermaps is
> > getting large (>1k lines; in this specific case, it's about 10k lines
> > large).
> >
> > The installation runs on a virtualized platform, using XEN. Postfinger
> > output is attached. I should also mention that, for various reasons,
> > it's not *easily* possible for me to simply upgrade the Postfix version.
> >
> >
> > Thank you very much,
> > Erik
>
> > postfinger - postfix configuration on Tue Jan 26 15:18:25 CET 2010
> > version: 1.30
> >
> > Warning: postfinger output may show private configuration information,
> > such as ip addresses and/or domain names which you do not want to show
> > to the public. If this is the case it is your responsibility to modify
> > the output to hide this private information. [Remove this warning with
> > the --nowarn option.]
> >
> > --System Parameters--
> > mail_version = 2.6.2-RC1
> > hostname = hostmail
> > uname = Linux hostmail 2.6.24-24-server #1 SMP Tue Jun 30 21:03:25 UTC 2009
> > i686 GNU/Linux
> >
> > --Packaging information--
> > looks like this postfix comes from deb package: postfix-2.6.2~rc1-1
> >
> > --main.cf non-default parameters--
> > alias_maps = hash:/etc/aliases
> > anvil_rate_time_unit = 30m
> > append_dot_mydomain = no
> > biff = no
> > bounce_queue_lifetime = 1h
> > broken_sasl_auth_clients = yes
> > content_filter = smtp-amavis:[127.0.0.1]:10024
> > header_checks = regexp:/etc/postfix/header_checks
> > local_recipient_maps = hash:/etc/postfix/local_rcpt_map
> > mailbox_size_limit = 0
> > mailbox_transport_maps = hash:/etc/postfix/mbox_transport
> > maximal_queue_lifetime = 6h
> > message_size_limit = 500000000
> > mydestination = localhost, $myhostname
> > myhostname = hostmail.XXXXXXXX.de
> > mynetworks = 127.0.0.0/8
> > queue_minfree = 1000000000
> > recipient_delimiter = +
> > relay_domains = hash:/etc/postfix/transport
> > relay_recipient_maps = regexp:/etc/postfix/usermaps
> > smtpd_banner = $myhostname ANTISPAM PROXY
> > smtpd_client_connection_rate_limit = 200
> > smtpd_client_restrictions = check_client_access
> > cidr:/etc/postfix/amavis_bypass_internal_warn, check_client_access
> > cidr:/etc/postfix/amavis_bypass_internal_filter, check_client_access
> > cidr:/etc/postfix/amavis_bypass_filter_smtpcrypt, check_client_access
> > cidr:/etc/postfix/amavis_bypass_filter, check_client_access
> > cidr:/etc/postfix/amavis_bypass_accept, check_client_access
> > cidr:/etc/postfix/amavis_bypass_internal_accept,
> > smtpd_data_restrictions = reject_unauth_pipelining,
> > smtpd_helo_required = yes
> > smtpd_recipient_restrictions = check_client_access
> > cidr:/etc/postfix/amavis_bypass_internal_accept, check_recipient_access
> > regexp:/etc/postfix/filter-quarantine.regexp, check_policy_service
> > inet:127.0.0.1:10040, permit_sasl_authenticated, permit_mynetworks,
> > reject_unauth_destination, reject_non_fqdn_sender,
> > reject_unknown_sender_domain, permit
> > smtpd_restriction_classes = rc_greylisting
> > smtpd_sasl_authenticated_header = yes
> > smtpd_sender_restrictions = permit_sasl_authenticated, check_sender_access
> > regexp:/etc/postfix/amavis_senderbypass_filter, permit_mynetworks, permit
> > smtpd_timeout = 60
> > transport_maps = hash:/etc/postfix/transport
> > virtual_gid_maps = static:114
> > virtual_mailbox_base = /var/quarantine
> > virtual_mailbox_limit = 1000000000
> > virtual_mailbox_maps = hash:/etc/postfix/virtual_mbox
> > virtual_uid_maps = static:106
> >
> > --master.cf--
> > 0.0.0.0:smtp inet n - - - 48 smtpd
> > pickup fifo n - - 60 1 pickup
> > -o content_filter=
> > cleanup unix n - - - 0 cleanup
> > qmgr fifo n - n 300 1 qmgr
> > tlsmgr unix - - - 1000? 1 tlsmgr
> > rewrite unix - - - - - trivial-rewrite
> > bounce unix - - - - 0 bounce
> > defer unix - - - - 0 bounce
> > trace unix - - - - 0 bounce
> > verify unix - - - - 1 verify
> > flush unix n - - 1000? 0 flush
> > proxymap unix - - n - - proxymap
> > smtp unix - - - - - smtp
> > relay unix - - - - - smtp
> > -o fallback_relay=
> > showq unix n - - - - showq
> > error unix - - - - - error
> > discard unix - - - - - discard
> > local unix - n n - - local
> > virtual unix - n n - - virtual
> > lmtp unix - - - - - lmtp
> > anvil unix - - - - 1 anvil
> > scache unix - - - - 1 scache
> > maildrop unix - n n - - pipe
> > flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
> > uucp unix - n n - - pipe
> > flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail
> > ($recipient)
> > ifmail unix - n n - - pipe
> > flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
> > bsmtp unix - n n - - pipe
> > flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender
> > $recipient
> > scalemail-backend unix - n n - 2 pipe
> > flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
> > ${nexthop} ${user} ${extension}
> > mailman unix - n n - - pipe
> > flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
> > ${nexthop} ${user}
> > smtp-amavis unix - - n - 16 smtp
> > -o smtp_data_done_timeout=1200
> > -o smtp_send_xforward_command=yes
> > -o disable_dns_lookups=yes
> > -o max_use=20
> > 127.0.0.1:10025 inet n - n - - smtpd
> > -o content_filter=
> > -o smtpd_restriction_classes=
> > -o smtpd_delay_reject=no
> > -o smtpd_client_restrictions=permit_mynetworks,reject
> > -o smtpd_helo_restrictions=
> > -o smtpd_sender_restrictions=
> > -o smtpd_recipient_restrictions=permit_mynetworks,reject
> > -o smtpd_data_restrictions=reject_unauth_pipelining
> > -o smtpd_end_of_data_restrictions=
> > -o mynetworks=127.0.0.0/8
> > -o smtpd_error_sleep_time=0
> > -o smtpd_soft_error_limit=1001
> > -o smtpd_hard_error_limit=1000
> > -o smtpd_client_connection_count_limit=0
> > -o smtpd_client_connection_rate_limit=0
> > -o smtpd_milters=
> > -o local_header_rewrite_clients=
> > -o local_recipient_maps=
> > -o relay_recipient_maps=
> > -o
> > receive_override_options=no_header_body_checks,no_unknown_recipient_checks
> > smtpcrypt unix - n n - - pipe
> > flags=Rq user=smtpcrypt argv=/usr/local/bin/smtpcrypt.pl
> > sasl${sasl_method} ${client_address} ${sender} ${recipient}
> > retry unix - - - - - error
> >
> > -- end of postfinger output --
>
>
> --
> adrian ilarion ciobanu
> adria...@ciobanu.name
> http://pub.mud.ro/~cia
> +40 788 319 497
