Re: problem with sendmail -XV - VERP expansion
Keld Simonsen: [ Charset ISO-8859-1 unsupported, converting... ] > On Sun, Jun 27, 2010 at 11:33:02AM -0400, Wietse Venema wrote: > > I checked out a few details, and the result is that turning on > > VERP, before sending to a mailing list alias, does not turn on VERP > > when delivering to the members of that mailing list (except in a > > very special case that involves only local recipients). > > > > It can (and should) be made to work outside that special case, but > > that requires a few changes to Postfix internals, and that will > > take more time than I have available now. > > > > There is a workaround to turn on VERP after the mailing list. For > > example, deliver mail to the mailing list alias with one Postfix > > instance, then deliver mail to the members of that list with a > > second Postfix instance that turns on VERP via one of the methods > > in message <20100626210934.27b7e1f3...@spike.porcupine.org> (this > > involves content filters, or the Postfix 2.7 smtpd_command_filter > > feature). > > maybe there should be a note on this in the documentetion, eg in > http://www.postfix.org/VERP_README.html > > Does this mean that the VERP facility has not been fully functional > at any time, and thus that there have not been that many attempts on > utilizing it before? During the discussion on this list on VERP there > were very few contributions from other people than Wietse. So the > lack of use could explain this. Currently. VERP does not survive local(8) alias expansion except perhaps in a very special case. > I thikn VERP could be a very useful thing to me - having spent many hours > trying to track down erroneous email addresses on the lists I administer. > > I wonder if it was generally possibly to set postfix to send envelope > return addresses for all outgoing messages, eg by a parameter somewhere > in master.cf ? Is that posssible, and would there be any problems > in doing that? With the command line: "sendmail -f senderaddr" With the SMTP protocol: MAIL FROM: Wietse
Re: OT: ldap schema
On Sat, Jul 03, 2010 at 02:15:53AM +0200, Fran Garcia wrote: > Basically the schema should : > > - Be OpenLDAP compatible Not a problem. > - Allow multidomain I don't know what this means. > - Host transports for each defined account / email address. This is not a good idea. Avoid using LDAP for transport lookups. Instead: - rewrite envelope recipients to an appropriate destination domain via virtual(5) (i.e. virtual_alias_maps). - explicitly set virtual_alias_domains (even if empty). - Map each destination domain to a suitable transport via an indexed file (Berkeley DB hash or btree, CDB, ...) > - Integrate with dovecot and/or cyrus-imapd. Postfix will happily use any schema in which lookup keys (typically email addresses) can be mapped to a result value (or list of values which are transformed to a comma-separated result string) by a query as explained in: http://www.postfix.org/ldap_table.5.html http://www.postfix.org/LDAP_README.html Postfix has no preferred LDAP schemas, it operates at a higher level of abstraction, i.e. virtual_alias_maps, transport_maps, ... which can be implemented via LDAP if you so choose. The mapping between an actual LDAP dataset and the conceptual Postfix key/value table is up to you. -- Viktor.
recipient_bcc_maps... noticing duplicate log entries when handing off to remote server
I've been watching this for a while and still not sure what could be causing it it or if its a known issue, but thought I'd pass it along here on the mailing list to see whatever one else thought. I use recipient_bcc_maps to bcc email to my personal account to Gmail. recipient_bcc contents: --- domin...@slackadelic.comdomin...@gmail.com postconf -n output: --- alias_database = hash:/etc/postfix/aliases alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf broken_sasl_auth_clients = yes command_directory = /usr/sbin/ config_directory = /etc/postfix content_filter = amavisfeed:[127.0.0.1]:10024 daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 disable_vrfy_command = yes fast_flush_domains = $relay_domains html_directory = no mail_owner = postfix mailq_path = /usr/sbin/mailq manpage_directory = /usr/man mydomain = slackadelic.com myhostname = mail.slackadelic.com mynetworks = 216.23.240.160, 74.207.254.75, 127.0.0.0/8 myorigin = $mydomain newaliases_path = /usr/sbin/newaliases postscreen_dnsbl_action = drop postscreen_dnsbl_sites = bl.ipv6.spameatingmonkey.net, bl.spameatingmonkey.net, zen.spamhaus.org postscreen_greet_action = drop postscreen_hangup_action = drop queue_directory = /var/spool/postfix readme_directory = no recipient_bcc_maps = hash:/etc/postfix/recipient_bcc recipient_delimiter = + relay_domains = mysql:/etc/postfix/mysql_virtual_relay_domains_maps.cf relay_recipient_maps = hash:/etc/postfix/slamd64_relay_recipients,hash:/etc/postfix/twister_relay_recipients sample_directory = /etc/postfix/sample sendmail_path = /usr/sbin/sendmail setgid_group = postdrop smtpd_banner = $myhostname ESMTP $mail_name smtpd_data_restrictions = permit_mynetworks, reject_unauth_pipelining smtpd_delay_reject = no smtpd_etrn_restrictions = permit_mynetworks, reject smtpd_helo_required = yes smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_client_hostname, reject_unknown_sender_domain, reject_unauth_destination, reject_unauth_pipelining, check_policy_service unix:private/spf, check_helo_access hash:/etc/postfix/helo_access, reject_non_fqdn_helo_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_invalid_helo_hostname, warn_if_reject reject_rbl_client 3625447584.geobl.spameatingmonkey.net, warn_if_reject reject_rhsbl_sender dsn.rfc-ignorant.org, reject_rbl_client bl.ipv6.spameatingmonkey.net, reject_rbl_client bl.spameatingmonkey.net, reject_rbl_client zen.spamhaus.org smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_starttls_timeout = 60s smtpd_tls_ccert_verifydepth = 1 smtpd_tls_cert_file = /etc/ssl/imap/mail.slackadelic.com.pem smtpd_tls_key_file = /etc/ssl/imap/private/mail.slackadelic.com.key smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may smtpd_tls_session_cache_timeout = 3600s soft_bounce = no tls_random_source = dev:/dev/urandom transport_maps = hash:/etc/postfix/transport unknown_client_reject_code = 554 unknown_local_recipient_reject_code = 550 virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf virtual_gid_maps = static:94 virtual_mailbox_base = /var/mail/virtual virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf virtual_mailbox_limit = 5120 virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf virtual_minimum_uid = 94 virtual_transport = virtual virtual_uid_maps = static:94 Relevant logs to the issue: --- http://pastebin.slackadelic.com/p/VUgFOC21.html Look at lines 25,26,27,28 as they are where the duplicate log entries are at. Definitely could be something with my setup, but this is the only time I see the duplicate is when the bcc map sends the email off-site. -Matt
Re: recipient_bcc_maps... noticing duplicate log entries when handing off to remote server
On 7/6/2010 10:11 AM, Matt Hayes wrote: > I've been watching this for a while and still not sure what could be > causing it it or if its a known issue, but thought I'd pass it along > here on the mailing list to see whatever one else thought. > Crap, forgot to add this in, my apologies! mail_version = 2.8-20100601 -Matt
Re: OT: ldap schema
This site uses LDAP for postfix/dovecot administration since about ten years. We use qmailControl.schema (to define the domains, which are accepted at this site) and qmail.schema (to define the mailboxes whithin these domains). suomi On 2010-07-06 15:58, Victor Duchovni wrote: On Sat, Jul 03, 2010 at 02:15:53AM +0200, Fran Garcia wrote: Basically the schema should : - Be OpenLDAP compatible Not a problem. - Allow multidomain I don't know what this means. - Host transports for each defined account / email address. This is not a good idea. Avoid using LDAP for transport lookups. Instead: - rewrite envelope recipients to an appropriate destination domain via virtual(5) (i.e. virtual_alias_maps). - explicitly set virtual_alias_domains (even if empty). - Map each destination domain to a suitable transport via an indexed file (Berkeley DB hash or btree, CDB, ...) - Integrate with dovecot and/or cyrus-imapd. Postfix will happily use any schema in which lookup keys (typically email addresses) can be mapped to a result value (or list of values which are transformed to a comma-separated result string) by a query as explained in: http://www.postfix.org/ldap_table.5.html http://www.postfix.org/LDAP_README.html Postfix has no preferred LDAP schemas, it operates at a higher level of abstraction, i.e. virtual_alias_maps, transport_maps, ... which can be implemented via LDAP if you so choose. The mapping between an actual LDAP dataset and the conceptual Postfix key/value table is up to you.
Debian package installation
I'm doing a custom install, and one of the packages in the install is postfix. Each time, it prompts me to select "no configuration" "Local use" etc. just after the package has been downloaded and right before it has been installed. (similar to the screen that shows up when you're asked to accept the sun-java6 license) I need a way to dodge it. Any ideas?
Re: recipient_bcc_maps... noticing duplicate log entries when handing off to remote server
On Tue, Jul 06, 2010 at 10:11:27AM -0400, Matt Hayes wrote: > I've been watching this for a while and still not sure what could be > causing it it or if its a known issue, but thought I'd pass it along > here on the mailing list to see whatever one else thought. > > I use recipient_bcc_maps to bcc email to my personal account to Gmail. > Relevant logs to the issue: > --- > > http://pastebin.slackadelic.com/p/VUgFOC21.html Look at lines > 25,26,27,28 as they are where the duplicate log entries are at. Please be kind to the people helping you and do not use pastebins, they are a pain in the butt to use. Post the logs, and obfuscate the local parts of any addresses you want to keep private: Jul 6 09:59:36 cyberslack postfix/smtpd[20468]: 366C57A17A: client=russian-caravan.cloud9.net[168.100.1.4] Jul 6 09:59:36 cyberslack postfix/cleanup[20474]: 366C57A17A: message-id=<20100706135843.gj5...@np305c2n2.ms.com> Jul 6 09:59:36 cyberslack postfix/qmgr[16355]: 366C57A17A: from=, size=5201, nrcpt=2 (queue active) Jul 6 09:59:37 cyberslack postfix-reinject/smtpd[20481]: 2EDEE7A22F: client=localhost[127.0.0.1] Jul 6 09:59:37 cyberslack postfix/cleanup[20474]: 2EDEE7A22F: message-id=<20100706135843.gj5...@np305c2n2.ms.com> Jul 6 09:59:37 cyberslack postfix/qmgr[16355]: 2EDEE7A22F: from=, size=5585, nrcpt=3 (queue active) Jul 6 09:59:37 cyberslack amavis[19206]: (19206-08) Passed CLEAN, [168.100.1.4] [170.74.93.68] -> ,, Message-ID: <20100706135843.gj5...@np305c2n2.ms.com>, Hits: -0.064, 934 ms Jul 6 09:59:37 cyberslack amavisfeed/smtp[20476]: 366C57A17A: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=2.5, delays=1.6/0.01/0.01/0.94, dsn=2.6.0, status=sent (250 2.6.0 Ok, id=19206-08, from MTA: 250 2.0.0 Ok: queued as 2EDEE7A22F) Jul 6 09:59:37 cyberslack amavisfeed/smtp[20476]: 366C57A17A: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=2.5, delays=1.6/0.01/0.01/0.94, dsn=2.6.0, status=sent (250 2.6.0 Ok, id=19206-08, from MTA: 250 2.0.0 Ok: queued as 2EDEE7A22F) Jul 6 09:59:37 cyberslack postfix/qmgr[16355]: 366C57A17A: removed Jul 6 09:59:37 cyberslack postfix/virtual[20482]: 2EDEE7A22F: to=, relay=virtual, delay=0.12, delays=0.07/0.02/0/0.02, dsn=2.0.0, status=sent (delivered to maildir) Jul 6 09:59:37 cyberslack postfix/smtp[20483]: 2EDEE7A22F: to=, relay=gmail-smtp-in.l.google.com[74.125.95.27]:25, delay=0.76, delays=0.07/0.05/0.29/0.34, dsn=2.0.0, status=sent (250 2.0.0 OK 1278424777 16si8703246ibr.79) Jul 6 09:59:37 cyberslack postfix/smtp[20483]: 2EDEE7A22F: to=, relay=gmail-smtp-in.l.google.com[74.125.95.27]:25, delay=0.76, delays=0.07/0.05/0.29/0.34, dsn=2.0.0, status=sent (250 2.0.0 OK 1278424777 16si8703246ibr.79) Jul 6 09:59:37 cyberslack postfix/qmgr[16355]: 2EDEE7A22F: removed > Definitely could be something with my setup, but this is the only time I > see the duplicate is when the bcc map sends the email off-site. Nothing wrong at all, there are no duplicate logs, delivery into amavis is logged, and delivery to the destination post-amavis is logged. -- Viktor.
Re: recipient_bcc_maps... noticing duplicate log entries when handing off to remote server
On 7/6/2010 10:43 AM, Victor Duchovni wrote: > On Tue, Jul 06, 2010 at 10:11:27AM -0400, Matt Hayes wrote: > >> I've been watching this for a while and still not sure what could be >> causing it it or if its a known issue, but thought I'd pass it along >> here on the mailing list to see whatever one else thought. >> >> I use recipient_bcc_maps to bcc email to my personal account to Gmail. > >> Relevant logs to the issue: >> --- >> >> http://pastebin.slackadelic.com/p/VUgFOC21.html Look at lines >> 25,26,27,28 as they are where the duplicate log entries are at. > > Please be kind to the people helping you and do not use pastebins, > they are a pain in the butt to use. Post the logs, and obfuscate > the local parts of any addresses you want to keep private: > Noted. > > Nothing wrong at all, there are no duplicate logs, delivery into amavis > is logged, and delivery to the destination post-amavis is logged. > Makes sense. Thanks. -Matt
Re: Debian package installation
On Tue, Jul 06, 2010 at 05:27:25PM +0300, Isaac Witmer wrote: > I'm doing a custom install, and one of the packages in the install is postfix. > Each time, it prompts me to select "no configuration" "Local use" etc. > just after the package has been downloaded and right before it has > been installed. (similar to the screen that shows up when you're asked > to accept the sun-java6 license) > > I need a way to dodge it. Any ideas? This is a Debian package management question, not a Postfix question. Please take it to a Debian list. -- Viktor.
Re: Debian package installation
Could you point me to the specific list you're referring to? On Tue, Jul 6, 2010 at 5:46 PM, Victor Duchovni wrote: > On Tue, Jul 06, 2010 at 05:27:25PM +0300, Isaac Witmer wrote: > >> I'm doing a custom install, and one of the packages in the install is >> postfix. >> Each time, it prompts me to select "no configuration" "Local use" etc. >> just after the package has been downloaded and right before it has >> been installed. (similar to the screen that shows up when you're asked >> to accept the sun-java6 license) >> >> I need a way to dodge it. Any ideas? > > This is a Debian package management question, not a Postfix question. > Please take it to a Debian list. > > -- > Viktor. >
Re: Debian package installation
google "debian forums" match your issue with the closest forum category and post the question there. -J On Tue, Jul 6, 2010 at 11:17 AM, Isaac Witmer wrote: > Could you point me to the specific list you're referring to? > > On Tue, Jul 6, 2010 at 5:46 PM, Victor Duchovni > wrote: > > On Tue, Jul 06, 2010 at 05:27:25PM +0300, Isaac Witmer wrote: > > > >> I'm doing a custom install, and one of the packages in the install is > postfix. > >> Each time, it prompts me to select "no configuration" "Local use" etc. > >> just after the package has been downloaded and right before it has > >> been installed. (similar to the screen that shows up when you're asked > >> to accept the sun-java6 license) > >> > >> I need a way to dodge it. Any ideas? > > > > This is a Debian package management question, not a Postfix question. > > Please take it to a Debian list. > > > > -- > >Viktor. > > >
Re: Postfix 2.7 for RHEL 5?
On 07/03/2010 01:27 PM, /dev/rob0 wrote: On Sat, Jul 03, 2010 at 02:53:44PM -0500, Stan Hoeppner wrote: Morten P.D. Stevens put forth on 7/3/2010 2:40 PM: Does anyone know backported Postfix 2.6.x or 2.7.x RPM packages for RHEL5? This binary rpm is for x86-64 only: http://ftp.wl0.org/official/2.7/RPMS-rhel5-x86_64/postfix-2.7.1-1.rhel5.x86_64.rpm You'll have to google more than I did to find an i386 binary rpm for 2.6.x or 2.7.x. I would suggest using a SRPM: http://ftp.wl0.org/official/2.7/SRPMS/postfix-2.7.1-1.src.rpm which can be configured and built as desired. Love to -- plus I'm dealing with not-64 bit machines -- but I can't find a RPM for tinycdb I feel comfortable with. All were circa 2002. Is this OK? What are others using? \\||/ Rod --
Fw: Fax problem
Hi I want to rewrite "From filed" from header,but only when To: Field is only numeric (fax type) It is: If To: nume...@domain.tld then From replace with f...@mydomain.tld Endif It is possible (Canical?) Thanks Gaby
Re: Debian package installation
Isaac Witmer wrote: > Could you point me to the specific list you're referring to? A good catchall is debian-u...@lists.debian.org where general discussion takes place. Bob
Re: Postfix 2.7 for RHEL 5?
On Tue, 06 Jul 2010 09:01:53 -0700, "Roderick A. Anderson" > Love to -- plus I'm dealing with not-64 bit machines -- but I can't find > a RPM for tinycdb I feel comfortable with. All were circa 2002. Is > this OK? What are others using? > > > \\||/ > Rod http://www.corpit.ru/mjt/tinycdb.html "Latest version is 0.77, released 31 Jan 2009, and can be found here. It can be built on systems using RedHat Package Manager (rpm) with -tb option to create installable .rpm package. On a Debian GNU/Linux system, the preferred way to install it is to use standard apt repository. For other versions of the package and pre-built rpms look here. " Guess you will manage now :-) -- Bas.
Re: Fw: Fax problem
On Tue, Jul 06, 2010 at 07:03:14PM +0300, Gaby L / AutoGlobus2000 SRL wrote: > I want to rewrite "From filed" from header,but only when To: Field > is only numeric (fax type) > It is: > If To: nume...@domain.tld then > From replace with f...@mydomain.tld > Endif Only via external content filters or milters. -- Viktor.
Re: Postfix 2.7 for RHEL 5?
On 07/06/2010 09:07 AM, Bas Mevissen wrote: On Tue, 06 Jul 2010 09:01:53 -0700, "Roderick A. Anderson" Love to -- plus I'm dealing with not-64 bit machines -- but I can't find a RPM for tinycdb I feel comfortable with. All were circa 2002. Is this OK? What are others using? \\||/ Rod http://www.corpit.ru/mjt/tinycdb.html "Latest version is 0.77, released 31 Jan 2009, and can be found here. It can be built on systems using RedHat Package Manager (rpm) with -tb option to create installable .rpm package. On a Debian GNU/Linux system, the preferred way to install it is to use standard apt repository. For other versions of the package and pre-built rpms look here. " Guess you will manage now :-) Thanks. That was the ticket. Rod --
Re: Fw: Fax problem
On Tue, Jul 6, 2010 at 12:14, Victor Duchovni wrote: > On Tue, Jul 06, 2010 at 07:03:14PM +0300, Gaby L / AutoGlobus2000 SRL wrote: > >> I want to rewrite "From filed" from header,but only when To: Field >> is only numeric (fax type) >> It is: >> If To: nume...@domain.tld then >> From replace with f...@mydomain.tld >> Endif > > Only via external content filters or milters. If it were possible to reconfigure any map to specify an alternate lookup key or key-tuple, in place of whatever default that map has, there would be a lot more power in that, including doing what Gaby L seems to want to do. I've had a couple other ideas in the past where "if only I could specify my own lookup key for this map" came to mind. But I had to give up on such ideas.
Re: Debian package installation
Isaac Witmer put forth on 7/6/2010 9:27 AM: > I'm doing a custom install, and one of the packages in the install is postfix. > Each time, it prompts me to select "no configuration" "Local use" etc. > just after the package has been downloaded and right before it has > been installed. (similar to the screen that shows up when you're asked > to accept the sun-java6 license) > > I need a way to dodge it. Any ideas? Yes. This is a helper script to ease setup burden. If you select "no configuration" you can then manually do whatever you want/need to with master.cf, main.cf, etc after system/package installation. Is the description "no configuration" not sufficiently explanatory? -- Stan
Re: Debian package installation
On Tue, Jul 6, 2010 at 10:27, Isaac Witmer wrote: > I'm doing a custom install, and one of the packages in the install is postfix. > Each time, it prompts me to select "no configuration" "Local use" etc. > just after the package has been downloaded and right before it has > been installed. (similar to the screen that shows up when you're asked > to accept the sun-java6 license) > > I need a way to dodge it. Any ideas? The package comes with two or more pre-packaged configurations to make it ready to go. Why not just use "no configuration" and later apply your own configuration. If you are trying to bypass the interactiveness of it so you don't get stopped at that choice, maybe you need an expect script (I've used pexpect with Python for various things, and was thinking of using it for this, too).
Re: Debian package installation
06.07.2010 20:58, Phil Howard wrote: > On Tue, Jul 6, 2010 at 10:27, Isaac Witmer wrote: >> I'm doing a custom install, and one of the packages in the install is >> postfix. >> Each time, it prompts me to select "no configuration" "Local use" etc. >> just after the package has been downloaded and right before it has >> been installed. (similar to the screen that shows up when you're asked >> to accept the sun-java6 license) >> >> I need a way to dodge it. Any ideas? > > The package comes with two or more pre-packaged configurations to make > it ready to go. Why not just use "no configuration" and later apply > your own configuration. > > If you are trying to bypass the interactiveness of it so you don't get > stopped at that choice, maybe you need an expect script (I've used > pexpect with Python for various things, and was thinking of using it > for this, too). This is becoming more and more off-topic for Postfix mailing list... there's debconf-set-selections command in Debian that is especially designed to pre-set answers to dpkg questions for non-interactive installations. There's no need to re-invent the wheel, it is here for a long time already and is working quite well. What you need is to install a package(s) in question on a test system and look at the debconf items of your interest. The raw data is stored in /var/cache/debconf/config.dat. But again, this has nothing to do with postfix, it's 100% debian question. In particular, read about how to do some non-interactive package installs in this distribution. /mjt
Re: DNS load-balancing two equal nexthops is not fair
On 06/30/2010 11:17 AM, Wietse Venema wrote: When sending mail via SMTP, Postfix randomizes the order of equal-preference server IP addresses. However, with SMTP connection caching enabled, the faster SMTP server will get more mail than the slower SMTP server. It seems you imply that disabling the connection cache will equalize the distribution. Or is it not that simple? Note: The systems are pretty fast and the connections are not slow either - one is local, the other is over a reasonably fast data link. -- Florin Andrei http://florin.myip.org/
Re: DNS load-balancing two equal nexthops is not fair
On Tue, Jul 06, 2010 at 11:21:19AM -0700, Florin Andrei wrote: > On 06/30/2010 11:17 AM, Wietse Venema wrote: >> When sending mail via SMTP, Postfix randomizes the order of >> equal-preference server IP addresses. >> >> However, with SMTP connection caching enabled, the faster SMTP >> server will get more mail than the slower SMTP server. > > It seems you imply that disabling the connection cache will equalize the > distribution. Or is it not that simple? No, disabling the cache will still leave a skewed distribution. Connection creation is uniform across the servers, but connection lifetime is much longer on the slow server, so its connection concurrency is much higher (potentially equal to the destination concurrency limit under suitable conditions, thus keeping the fast servers essentially idle). A time-based cache is the fairness mechanism that keeps connection lifetimes uniform across the servers, which ensures non-starvation of fast servers, and avoids futher overload of (congested) slow servers. > Note: The systems are pretty fast and the connections are not slow either - > one is local, the other is over a reasonably fast data link. The is not always hitting the fan, otherwise the fan would be off. :-) -- Viktor.
Re: DNS load-balancing two equal nexthops is not fair
On 07/06/2010 11:30 AM, Victor Duchovni wrote:
No, disabling the cache will still leave a skewed distribution. Connection
creation is uniform across the servers, but connection lifetime is much
longer on the slow server, so its connection concurrency is much higher
(potentially equal to the destination concurrency limit under suitable
conditions, thus keeping the fast servers essentially idle).
A time-based cache is the fairness mechanism that keeps connection
lifetimes uniform across the servers, which ensures non-starvation
of fast servers, and avoids futher overload of (congested) slow servers.
I see.
I realize that email delivery is not a trivial problem, but it seems
baffling that a seemingly simple task ("fair" volume-based load
balancing between transports) is so hard to achieve.
A very dumb algorithm should accomplish it: single-threaded delivery (no
concurrency), a "voluntary" (sender-side) limit of N messages delivered
per connection, then reconnect. DNS randomization should then do the
trick. If the network and the servers are fast (and they are, in my
case), this shouldn't slow down the delivery too much (in fact, a small
speed decrease might be beneficial).
I think I know how to eliminate concurrency, but I'm lacking a
volume-based limit for the connections.
I'll keep looking for a solution.
--
Florin Andrei
http://florin.myip.org/
Re: DNS load-balancing two equal nexthops is not fair
On Tue, Jul 06, 2010 at 12:10:41PM -0700, Florin Andrei wrote:
> I realize that email delivery is not a trivial problem, but it seems
> baffling that a seemingly simple task ("fair" volume-based load balancing
> between transports) is so hard to achieve.
If you want to deliver the same number of messages to each server,
regardless of server performance, (message-count fairness, rather than
concurrency fairness), and suffer high latency when a slow server starts
to impede message flow, then turning off the cache will indeed give you
roughly uniform message distribution:
- *New* connections are distributed uniformly
- There is at most one delivery per connection
- Hence messages are distributed uniformly
However, concurrency will not be distributed uniformly, and a slow
server will account for most or all of the concurrency, ensuring a
high average latency even when alternative servers are sitting idle.
> I'll keep looking for a solution.
What negative symptoms are your systems exhibiting?
What *real* problem are you trying to solve?
--
Viktor.
spam that does get through looks normal.
I have now went through my config so I will post it if needed. What I'm facing now is spam that looks normal. Looks like a reject but is not in some cases. The problem is that since these e-mails are delivered to the user account. I really don't have an example to post from the q. I use postini, mailscanner, that uses clamav and spamassasian. That does a good job but I still get spam through. Even on top of using outlook 2003 / 2007 spam filter. The current small batch of say 5 messages looked like rejects. Sure I can look at the header and see what server they are comming from. In fact some of the messages are from postmaster at whatever server. But it does not matter. This spam slips through and I'm told about it. I cannot tell them to black list the address since it keeps changing. I think I need a better spam filter or to change some settings. But how do you kill mail that looks normal? (I think I asked this before. So plz forgive me if I did. Perhaps this plea for help for have some new ideas) Josh -- This message has been scanned for viruses and dangerous content by Mychoice, and is believed to be clean.
Re: DNS load-balancing two equal nexthops is not fair
On 07/06/2010 12:27 PM, Victor Duchovni wrote: If you want to deliver the same number of messages to each server, regardless of server performance, (message-count fairness, rather than concurrency fairness), and suffer high latency when a slow server starts to impede message flow, then turning off the cache will indeed give you roughly uniform message distribution: - *New* connections are distributed uniformly - There is at most one delivery per connection - Hence messages are distributed uniformly However, concurrency will not be distributed uniformly, and a slow server will account for most or all of the concurrency, ensuring a high average latency even when alternative servers are sitting idle. That's fine. One transport is on the local network, the other is across a data link that would have been considered "as fast as local" not too long ago. Both servers are modern fast hardware. Both are highly available from the p.o.v. of the machines generating the emails. Even if one of them disappears, so what, the other will just magically take over and at most we're not worse off than before. The "slow" server, therefore, is not that "slow". It's just different enough (latency, mostly) to tip over the sensitive delivery algorithm, which seems to be fine-tuned for Internet conditions, rather than local or near-local networks. From what you're saying, it appears that single-threaded delivery is unnecessary - the email "generators" will simply hit the upper connection limit and stay near it, with newly released slots being occupied by either one relay or the other at random. That should ensure a "fair" distribution, I think. What negative symptoms are your systems exhibiting? What *real* problem are you trying to solve? The real problem was described in the other big thread I started recently: delivery to a certain big popular email provider is exceedingly slow. We have a pretty small delivery window between the moment the messages are created and the moment they should be available to the users - that's not a problem with all the other providers (heck, Gmail for instance seems to absorb emails way faster than we can send them - this even while their anti-spam filters seem at once more fair and more effective than the other providers'). We already did long time ago some of the stuff you indicated (the spam feedback loop, etc.) and have started a while ago working on the rest (whitelisting, etc.) which is supposed to get us out of the red zone. But *meanwhile* I have to make the best out of a tricky set of mutually-exclusive constraints. Having multiple exit points seems to improve the overall delivery speed - this is true even right now, when distribution is skewed to the faster server 4:1. My estimate is, a near-1:1 distribution would actually fix our time-constraint problem even before whitelisting. So you see how this is kind of a big incentive to get it done. -- Florin Andrei http://florin.myip.org/
Re: DNS load-balancing two equal nexthops is not fair
On Tue, Jul 06, 2010 at 01:00:14PM -0700, Florin Andrei wrote: > Having multiple exit points seems to improve the overall delivery speed - > this is true even right now, when distribution is skewed to the faster > server 4:1. My estimate is, a near-1:1 distribution would actually fix our > time-constraint problem even before whitelisting. So you see how this is > kind of a big incentive to get it done. So you have multiple exit points with non-uniform latency, but the more severe congestion is downstream, so you want to load the exit points uniformly. Yes, the solution is to disable the connection cache, and set reasonably low connection and helo timeouts in the transport feeding the two exit points, so that when one is down and non-responsive (no TCP reset), you don't suffer excessive hand-off latency for 50% of deliveries. master.cf: transp unix ... smtp -o smtp_connect_timeout=$_connect_timeout -o smtp_helo_timeout=$_helo_timeout main.cf: # default is 30s transp_connect_timeout = 2s # default is 300s transp_helo_timeout = 30s -- Viktor.
Re: spam that does get through looks normal.
On Tue, Jul 6, 2010 at 16:10, Josh Cason wrote: > I have now went through my config so I will post it if needed. What I'm > facing now is spam that looks normal. Looks like a reject but is not in some > cases. The problem is that since these e-mails are delivered to the user > account. I really don't have an example to post from the q. I use postini, > mailscanner, that uses clamav and spamassasian. That does a good job but I > still get spam through. Even on top of using outlook 2003 / 2007 spam > filter. The current small batch of say 5 messages looked like rejects. Sure > I can look at the header and see what server they are comming from. In fact > some of the messages are from postmaster at whatever server. But it does not > matter. This spam slips through and I'm told about it. I cannot tell them to > black list the address since it keeps changing. I think I need a better spam > filter or to change some settings. But how do you kill mail that looks > normal? Are these so normal that they don't even look alike? I can't imagine humans writing decent message content on spammer scales, so they must have some better AI these days (probably using the same anti-spam filters to train their own spam generators). -- sHiFt HaPpEnS!
Re: Postfix as an outbound mail gateway
I'm sorry, Was my question unclear? Or just too ridiculously simple for this group? I think I may be able to figure out the allowed relay part... But I'm just looking for some reassurance that I'm not going to disrupt the existing Mailman List Manager already running on this system. The outbound SMTP traffic will be minimal at most. Jeff :) On Tue, Jun 29, 2010 at 12:05 PM, Jeff Bernier wrote: > Hello all, > > I have a system running Mailman for our lists, and Postfix. > Recently, we retired our in-house email system to go with a hosted email > system off campus. Because of this, we no longer need, and will also be > decommissioning our expensive email anti-spam/anti-virus system which was > also our SMTP gateway. > > We would like to continue to restrict outbound mail to one smtp mail > gateway on our network. I would like to use Postfix on our Mailing List > server as this outbound mail gateway. I do not need it to do authentication > of senders, but rather specify a small group of allowed host senders. > > My question is... Can this be easily done without disturbing Mailman list > traffic? > > Thanks, > Jeff > > -- > Jeff Bernier > > Office of Information Technology > Rhode Island School of Design > 401.454.6168 > -- Jeff Bernier Office of Information Technology Rhode Island School of Design 401.454.6168
Re: Postfix as an outbound mail gateway
On Tue, Jul 06, 2010 at 04:31:21PM -0400, Jeff Bernier wrote: > I'm sorry, > > Was my question unclear? Or just too ridiculously simple for this group? Neither, it was too general. You need to ask more specific questions. > > My question is... Can this be easily done without disturbing Mailman list > > traffic? Would "yes" really help you? If not, and given that the question is a yes/no question, it becomes clear that you need to ask something more concrete. -- Viktor.
re: spam that does get through looks normal.
No the message is different. Like this time around they look like this: This is an automatically generated Delivery Status Notification. Delivery to the following recipients failed. authentical...@raisley.com Final-Recipient: rfc958;authentical...@raisley.com Action: failed Status: 1.2.0 I prefer not keeping a long list of block. I would like to stop this garbage before it gets to me. The domain and mail address changes though. Josh -- This message has been scanned for viruses and dangerous content by Mychoice, and is believed to be clean.
Re: OT: ldap schema
On Tue, Jul 6, 2010 at 15:58, Victor Duchovni wrote: > On Sat, Jul 03, 2010 at 02:15:53AM +0200, Fran Garcia wrote: > >> Basically the schema should : >> >> - Be OpenLDAP compatible > > Not a problem. > >> - Allow multidomain > > I don't know what this means. Hi Viktor, thanks for your reply. This means "be able to hold several virtual domains as destination". Think of an ISP configuring a shared email platform for several domains / customers. Ideally those domains would be held in LDAP as well. (I've seen the qmail.schema and apparently is only ready fo one single domain). >> - Host transports for each defined account / email address. > > This is not a good idea. Avoid using LDAP for transport lookups. > Instead: > > - rewrite envelope recipients to an appropriate destination > domain via virtual(5) (i.e. virtual_alias_maps). > > - explicitly set virtual_alias_domains (even if empty). > > - Map each destination domain to a suitable transport via > an indexed file (Berkeley DB hash or btree, CDB, ...) The rationale for requesting this was "how do I grow if I have say 100k accounts in a single domain and I want to spread the load on several backend servers". As per your description, that would be handled like : us...@example.org -> us...@internal_backendx.example.org ? >> - Integrate with dovecot and/or cyrus-imapd. > > Postfix will happily use any schema in which lookup keys > (typically email addresses) can be mapped to a result > value (or list of values which are transformed to a comma-separated > result string) by a query as explained in: > > http://www.postfix.org/ldap_table.5.html > http://www.postfix.org/LDAP_README.html > > Postfix has no preferred LDAP schemas, it operates at a higher level of > abstraction, i.e. virtual_alias_maps, transport_maps, ... which can be > implemented via LDAP if you so choose. The mapping between an actual > LDAP dataset and the conceptual Postfix key/value table is up to you. Thanks for the links :-) . I already came across the "postfix adapts to any ldap schema" but, since I'm starting with ldap and not very familiar with all the concepts, I wanted to get some reall ife examples of actual schemas people are using. cheers
Re: OT: ldap schema
On 07/06/2010 04:22 PM, Fran Garcia wrote: > On Tue, Jul 6, 2010 at 15:58, Victor Duchovni wrote: >> On Sat, Jul 03, 2010 at 02:15:53AM +0200, Fran Garcia wrote: FWITW, I've used this as a reference in the past. After you build a few of these systems, They become quite easy. http://phamm.org/ kind regards, Terry
Re: spam that does get through looks normal.
On 7/6/2010 4:51 PM, Josh Cason wrote: No the message is different. Like this time around they look like this: This is an automatically generated Delivery Status Notification. Delivery to the following recipients failed. authentical...@raisley.com Final-Recipient: rfc958;authentical...@raisley.com Action: failed Status: 1.2.0 I prefer not keeping a long list of block. I would like to stop this garbage before it gets to me. The domain and mail address changes though. Josh Start here: http://www.postfix.org/BACKSCATTER_README.html And read up on the VBounce rules in SpamAssassin. Since you're using an outside mail filtering service, RBLs won't help.
Re: OT: ldap schema
On Tue, Jul 06, 2010 at 11:22:47PM +0200, Fran Garcia wrote: > >> - Allow multidomain > > > > I don't know what this means. > > Hi Viktor, thanks for your reply. > > This means "be able to hold several virtual domains as destination". > Think of an ISP configuring a shared email platform for several > domains / customers. Ideally those domains would be held in LDAP as > well. (I've seen the qmail.schema and apparently is only ready fo one > single domain). Postfix supports multiple domains not via pre-fab LDAP schemas, but via decisions about local and virtual users as described in http://www.postfix.org/VIRTUAL_README.html you can implement "virtual alias" or "virtual mailbox" users in as many domains as you wish, via any LDAP schema that contains the required address -> value (either address of mailbox path) mappings. > >> - Host transports for each defined account / email address. > > > > This is not a good idea. Avoid using LDAP for transport lookups. > > Instead: > > > > ? ?- rewrite envelope recipients to an appropriate destination > > ? ? ?domain via virtual(5) (i.e. virtual_alias_maps). > > > > ? ?- explicitly set virtual_alias_domains (even if empty). > > > > ? ?- Map each destination domain to a suitable transport via > > ? ? ?an indexed file (Berkeley DB hash or btree, CDB, ...) > > The rationale for requesting this was "how do I grow if I have say > 100k accounts in a single domain and I want to spread the load on > several backend servers". As per your description, that would be > handled like : us...@example.org -> > us...@internal_backendx.example.org ? Yes. > >> - Integrate with dovecot and/or cyrus-imapd. > > > > Postfix will happily use any schema in which lookup keys > > (typically email addresses) can be mapped to a result > > value (or list of values which are transformed to a comma-separated > > result string) by a query as explained in: > > > > ? ?http://www.postfix.org/ldap_table.5.html > > ? ?http://www.postfix.org/LDAP_README.html > > > > Postfix has no preferred LDAP schemas, it operates at a higher level of > > abstraction, i.e. virtual_alias_maps, transport_maps, ... ?which can be > > implemented via LDAP if you so choose. The mapping between an actual > > LDAP dataset and the conceptual Postfix key/value table is up to you. > > Thanks for the links :-) . I already came across the "postfix adapts > to any ldap schema" but, since I'm starting with ldap and not very > familiar with all the concepts, I wanted to get some reall ife > examples of actual schemas people are using. Design the Postfix configuration first, and the LDAP schema second. -- Viktor.
Re: Postfix.org SPF
junkyardma...@verizon.net a écrit : > Yahoo has ulterior motives? They wish to push their domain keys. > Others probably likewise have ulterior motives. > Do you also oppose SPF, and if so what is your motives? I will repeat myself: this is not the place to discuss SPF. SPF has been debated to death here and elsewhere, and those discussions brought nothing useful. It is your right to believe in the SPF God, but please don't preach for your religion here. if you want to know what I think about SPF, I'll invite you to search the archives of this list and the spamassassin list. With all due respect, I won't do that for you. if you're curious, I am not for nor against SPF. I take it as easily as: if everybody uses SPF, I have no choice but use SPF. until then, I don't care.
THREAD DEAD [Was: Postfix.org SPF]
didn't see Wietse message before sending. so please ignore my previous post. (sigh, there is no "get my post back" in email :). mouss a écrit : > junkyardma...@verizon.net a écrit : >> Yahoo has ulterior motives? They wish to push their domain keys. >> Others probably likewise have ulterior motives. >> Do you also oppose SPF, and if so what is your motives? > > > I will repeat myself: this is not the place to discuss SPF. SPF has been > debated to death here and elsewhere, and those discussions brought > nothing useful. It is your right to believe in the SPF God, but please > don't preach for your religion here. > > > if you want to know what I think about SPF, I'll invite you to search > the archives of this list and the spamassassin list. With all due > respect, I won't do that for you. > > if you're curious, I am not for nor against SPF. I take it as easily as: > if everybody uses SPF, I have no choice but use SPF. until then, I don't > care.
Re: spam that does get through looks normal.
Josh Cason a écrit : > I have now went through my config so I will post it if needed. What I'm > facing now is spam that looks normal. Looks like a reject but is not in > some cases. The problem is that since these e-mails are delivered to the > user account. I really don't have an example to post from the q. I use > postini, mailscanner, that uses clamav and spamassasian. That does a > good job but I still get spam through. Even on top of using outlook 2003 > / 2007 spam filter. The current small batch of say 5 messages looked > like rejects. Sure I can look at the header and see what server they are > comming from. In fact some of the messages are from postmaster at > whatever server. But it does not matter. This spam slips through and I'm > told about it. I cannot tell them to black list the address since it > keeps changing. I think I need a better spam filter or to change some > settings. But how do you kill mail that looks normal? > > (I think I asked this before. So plz forgive me if I did. Perhaps this > plea for help for have some new ideas) > so you're using postini but still have a spam problem? hmmm. do you accept mail from anything but postini servers? if so, why? ... please give a detailed explanation of your setup, and show your configuration (postconf -n).
Re: Debian package installation
Thanks Bob. I wasn't sure if Victor had a specific list in mind. It's not as if this is the first place I came. On Tue, Jul 6, 2010 at 7:06 PM, Bob Proulx wrote: > Isaac Witmer wrote: >> Could you point me to the specific list you're referring to? > > A good catchall is debian-u...@lists.debian.org where general > discussion takes place. > > Bob >
Re: Debian package installation
I would like to apologize for hijacking this mailing list, I didn't realize it would be quite so off topic. I was having trouble finding the answer in all the usual places. After almost writing a response, I've almost found the answer (haven't tested it yet) by searching for: postfix debconf-set-selections Good luck to others that need this. On Tue, Jul 6, 2010 at 8:43 PM, Michael Tokarev wrote: > 06.07.2010 20:58, Phil Howard wrote: >> On Tue, Jul 6, 2010 at 10:27, Isaac Witmer wrote: >>> I'm doing a custom install, and one of the packages in the install is >>> postfix. >>> Each time, it prompts me to select "no configuration" "Local use" etc. >>> just after the package has been downloaded and right before it has >>> been installed. (similar to the screen that shows up when you're asked >>> to accept the sun-java6 license) >>> >>> I need a way to dodge it. Any ideas? >> >> The package comes with two or more pre-packaged configurations to make >> it ready to go. Why not just use "no configuration" and later apply >> your own configuration. >> >> If you are trying to bypass the interactiveness of it so you don't get >> stopped at that choice, maybe you need an expect script (I've used >> pexpect with Python for various things, and was thinking of using it >> for this, too). > > This is becoming more and more off-topic for Postfix mailing list... > > there's debconf-set-selections command in Debian that is especially > designed to pre-set answers to dpkg questions for non-interactive > installations. There's no need to re-invent the wheel, it is here > for a long time already and is working quite well. What you need > is to install a package(s) in question on a test system and look > at the debconf items of your interest. The raw data is stored > in /var/cache/debconf/config.dat. > > But again, this has nothing to do with postfix, it's 100% debian > question. In particular, read about how to do some non-interactive > package installs in this distribution. > > /mjt >
