Catch all deliveries
Hi, I'm changing postfix local delivery to LMTP dovecot delivery and I'm have some doubts: Before to move to LMTP I have something like this: virtual_mailbox_maps: h...@example.com maildir1/ ad...@example.com maildir2/ @example.com maildir1/ This works ok with "@example.com" as catchall entry. But when move to LMTP, unexisting accounts start to produce error on dovecot, and I add cathcall accounts on aliases: virtual_mailbox_maps: h...@example.com maildir1/ ad...@example.com maildir2/ virtual_alias_maps: @example.com h...@example.com But with this configuration, catchall is working for ALL mail accounts, not only for unexisting accounts. Next test should be test to add all-accounts to alias maps: virtual_mailbox_maps: h...@example.com maildir1/ ad...@example.com maildir2/ virtual_alias_maps @example.com h...@example.com h...@example.com h...@example.com ad...@example.com ad...@example.com .. This is the correct way? is really necessary to duplicate all existing account on aliases with A A keyvalues? Isn't there a cleaner way to only match the non-existent ones?
Re: Catch all deliveries
On Tue, Feb 22, 2022 at 10:16:26AM +0100, Víctor Rubiella Monfort wrote: > Next test should be test to add all-accounts to alias maps: > > virtual_mailbox_maps: >h...@example.com maildir1/ >ad...@example.com maildir2/ > > virtual_alias_maps >@example.com h...@example.com >h...@example.com h...@example.com >ad...@example.com ad...@example.com > > .. > > This is the correct way? Yes, but with all those rewrites in virtual alias maps, you no longer need the virtual_mailbox_maps entries. They'd only be used for recipient validation, for which the virtual alias entries are sufficient. > Is t really necessary to duplicate all existing account on aliases > with A A keyvalues? No, you can leave virtual_mailbox_maps empty if you're not using the virtual(8) delivery agent, and using lmtp(8) to Dovecot instead. > Isn't there a cleaner way to only match the non-existent ones? No, but duplication is not needed, and if it were, you could always use "make" or similar to construct one of the tables from the other. -- Viktor.
Re: Catch all deliveries
Viktor Dukhovni:
> On Tue, Feb 22, 2022 at 10:16:26AM +0100, V?ctor Rubiella Monfort wrote:
>
> > Next test should be test to add all-accounts to alias maps:
> >
> > virtual_mailbox_maps:
> >h...@example.com maildir1/
> >ad...@example.com maildir2/
> >
> > virtual_alias_maps
> >@example.com h...@example.com
> >h...@example.com h...@example.com
> >ad...@example.com ad...@example.com
> >
> > ..
> >
> > This is the correct way?
>
> Yes, but with all those rewrites in virtual alias maps, you no longer
> need the virtual_mailbox_maps entries. They'd only be used for
> recipient validation, for which the virtual alias entries are
> sufficient.
>
> > Is t really necessary to duplicate all existing account on aliases
> > with A A keyvalues?
>
> No, you can leave virtual_mailbox_maps empty if you're not using the
> virtual(8) delivery agent, and using lmtp(8) to Dovecot instead.
>
> > Isn't there a cleaner way to only match the non-existent ones?
>
> No, but duplication is not needed, and if it were, you could always
> use "make" or similar to construct one of the tables from the other.
For a solution without email addresses in virtual_xx_maps, see
https://doc.dovecot.org/configuration_manual/howto/postfix_dovecot_lmtp/
This uses reject_unverified_recipient to discover what users exist.
The webpage is not complete, but this may help:
virtual_mailbox_domains = example.com
virtual_mailbox_maps = inline:{{@example.com = whatever}}
virtual_transport = lmtp:unix:private/dovecot-lmtp
smtpd_recipient_restrictions =
...
reject_unauth_destination,
reject_unverified_recipient
Corrections are welcome.
Wietse
Encoding problem when talking to postgresql
Hello, I am trying to set up postgresql-based virtual domains. I am on FreeBSD 13. When I try to test my domain mapping, I get a strange error about character encodings: # postmap -v -q example.org pgsql:/usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf postmap: name_mask: ipv4 postmap: name_mask: host postmap: inet_addr_local: configured 2 IPv4 addresses postmap: been_here: /32: 0 postmap: been_here: 127.0.0.1/32: 0 postmap: mynetworks_core: /32 127.0.0.1/32 postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: user = mailuser postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: password = postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: dbname = mailserver postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: result_format = %s postmap: cfg_get_int: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: expansion_limit = 0 postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: query = SELECT 1 FROM virtual_domains WHERE name='%s' postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: domain = postmap: cfg_get_str: /usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: hosts = 127.0.0.1 postmap: dict_open: pgsql:/usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf postmap: dict_pgsql_get_active: attempting to connect to host 127.0.0.1 postmap: dict_pgsql: successful connection to host 127.0.0.1 postmap: warning: dict_pgsql: cannot set the encoding to LATIN1, skipping 127.0.0.1 postmap: fatal: table pgsql:/usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf: query error: Application error I have enabled UTF8 in postfix as per the instructions here: http://www.postfix.org/SMTPUTF8_README.html#enabling , which you can see in the following config info: # postconf -n command_directory = /usr/local/sbin compatibility_level = 3.6 daemon_directory = /usr/local/libexec/postfix data_directory = /var/db/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 html_directory = /usr/local/share/doc/postfix inet_protocols = ipv4 mail_owner = postfix mailq_path = /usr/local/bin/mailq manpage_directory = /usr/local/man meta_directory = /usr/local/libexec/postfix mynetworks_style = host newaliases_path = /usr/local/bin/newaliases queue_directory = /var/spool/postfix readme_directory = /usr/local/share/doc/postfix sample_directory = /usr/local/etc/postfix sendmail_path = /usr/local/sbin/sendmail setgid_group = maildrop shlib_directory = /usr/local/lib/postfix smtputf8_enable = yes unknown_local_recipient_reject_code = 550 virtual_mailbox_domains = pgsql:/usr/local/etc/postfix/pgsql-virtual-mailbox-domains.cf How do I get postfix to speak UTF8 instead of LATIN1?
Re: Encoding problem when talking to postgresql
On Tue, Feb 22, 2022 at 01:41:00PM -0800, Nathan Van Ymeren wrote:
> I have enabled UTF8 in postfix as per the instructions here:
> http://www.postfix.org/SMTPUTF8_README.html#enabling , which you can
> see in the following config info:
This enables UTF8 in SMTP, but does not presently affect the PgSQL
driver:
src/global/dict_pgsql.c:
...
#ifdef SNAPSHOT
if (PQsetClientEncoding(host->db, "UTF8") != 0) {
msg_warn("dict_pgsql: cannot set the encoding to UTF8, skipping %s",
host->hostname);
plpgsql_down_host(host);
return;
}
#else
/*
* XXX Postfix does not send multi-byte characters. The following piece
* of code is an explicit statement of this fact, and the database
server
* should not accept multi-byte information after this point.
*/
if (PQsetClientEncoding(host->db, "LATIN1") != 0) {
msg_warn("dict_pgsql: cannot set the encoding to LATIN1, skipping
%s",
host->hostname);
plpgsql_down_host(host);
return;
}
#endif
...
So UTF8 is only used in development snapshots and not in release builds.
This code needs to be made either conditional on EAI (UTF8) being
enabled, or a new user-settable PgSQL table parameter.
> How do I get postfix to speak UTF8 instead of LATIN1?
You can run a snapshot build...
--
Viktor.
Re: Encoding problem when talking to postgresql
On Tue, Feb 22, 2022, 14:29 Viktor Dukhovni
wrote:
> On Tue, Feb 22, 2022 at 01:41:00PM -0800, Nathan Van Ymeren wrote:
>
> > I have enabled UTF8 in postfix as per the instructions here:
> > http://www.postfix.org/SMTPUTF8_README.html#enabling , which you can
> > see in the following config info:
>
> This enables UTF8 in SMTP, but does not presently affect the PgSQL
> driver:
>
> src/global/dict_pgsql.c:
> ...
> #ifdef SNAPSHOT
> if (PQsetClientEncoding(host->db, "UTF8") != 0) {
> msg_warn("dict_pgsql: cannot set the encoding to UTF8,
> skipping %s",
> host->hostname);
> plpgsql_down_host(host);
> return;
> }
> #else
>
> /*
> * XXX Postfix does not send multi-byte characters. The following
> piece
> * of code is an explicit statement of this fact, and the database
> server
> * should not accept multi-byte information after this point.
> */
> if (PQsetClientEncoding(host->db, "LATIN1") != 0) {
> msg_warn("dict_pgsql: cannot set the encoding to LATIN1,
> skipping %s",
> host->hostname);
> plpgsql_down_host(host);
> return;
> }
> #endif
> ...
>
> So UTF8 is only used in development snapshots and not in release builds.
> This code needs to be made either conditional on EAI (UTF8) being
> enabled, or a new user-settable PgSQL table parameter.
>
> > How do I get postfix to speak UTF8 instead of LATIN1?
>
> You can run a snapshot build...
>
> --
> Viktor.
>
I'm perplexed because this exact setup ran on Debian (which certainly
doesn't package snapshots).
It's only on FreeBSD where I run into trouble.
Re: Encoding problem when talking to postgresql
On Tue, Feb 22, 2022 at 02:32:29PM -0800, Nathan Van Ymeren wrote: > > So UTF8 is only used in development snapshots and not in release builds. > > This code needs to be made either conditional on EAI (UTF8) being > > enabled, or a new user-settable PgSQL table parameter. > > > > > How do I get postfix to speak UTF8 instead of LATIN1? > > > > You can run a snapshot build... > > I'm perplexed because this exact setup ran on Debian (which certainly > doesn't package snapshots). > > It's only on FreeBSD where I run into trouble. Perhaps on Debian your Postges database had a LATIN1 server encoding. And the FreeBSD 13 build defaulted the database to UTF8? You'd need to examine the Debian source package and your database to confirm. -- Viktor.
Re: Encoding problem when talking to postgresql
On Tue, Feb 22, 2022, 14:49 Viktor Dukhovni wrote: > On Tue, Feb 22, 2022 at 02:32:29PM -0800, Nathan Van Ymeren wrote: > > > > So UTF8 is only used in development snapshots and not in release > builds. > > > This code needs to be made either conditional on EAI (UTF8) being > > > enabled, or a new user-settable PgSQL table parameter. > > > > > > > How do I get postfix to speak UTF8 instead of LATIN1? > > > > > > You can run a snapshot build... > > > > I'm perplexed because this exact setup ran on Debian (which certainly > > doesn't package snapshots). > > > > It's only on FreeBSD where I run into trouble. > > Perhaps on Debian your Postges database had a LATIN1 server encoding. > And the FreeBSD 13 build defaulted the database to UTF8? > > You'd need to examine the Debian source package and your database to > confirm. > > -- > Viktor. > Hi Viktor, My database on Debian is certainly UTF-8. I will look into the source package. >
Patch: Encoding problem when talking to postgresql
Viktor Dukhovni:
> On Tue, Feb 22, 2022 at 01:41:00PM -0800, Nathan Van Ymeren wrote:
>
> > I have enabled UTF8 in postfix as per the instructions here:
> > http://www.postfix.org/SMTPUTF8_README.html#enabling , which you can
> > see in the following config info:
>
> This enables UTF8 in SMTP, but does not presently affect the PgSQL
> driver:
>
> src/global/dict_pgsql.c:
> ...
> #ifdef SNAPSHOT
There are two #ifdef SNAPSHOT blocks.
- The one in dict_pgsql_lookup() returns not found when SMTPUTF8
is enabled, but a query is not valid UTF8.
- The one in plpgsql_connect_single() sets the PgSQL client encoding
to UTF8, and if SNAPSHOT is not defined, leaves it at LATIN1
encoding which is what was used historically.
How do we avoid unexpected breakages in a stable release?
I suggest that we leave the encoding at LATIN1 when SMTPUTF8 is
disabled. That is historical behavior and that minimizes the risk
of breakages.
We can then use the UTF8 encoding when SMTPUTF8 is enabled. It is
the only encoding that makes sense with SMTPUTF8.
And we can ruminiate about what encoding should be used when SMTPUTF8
is disabled. There is no urgent need to change that now in a stable
release.
Wietse
Classical diff, for readability.
*** ./src/global/dict_pgsql.c- 2018-08-27 17:54:59.0 -0400
--- ./src/global/dict_pgsql.c 2022-02-22 19:03:57.858998431 -0500
***
*** 348,354
/*
* Don't frustrate future attempts to make Postfix UTF-8 transparent.
*/
- #ifdef SNAPSHOT
if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
&& !valid_utf8_string(name, strlen(name))) {
if (msg_verbose)
--- 348,353
***
*** 356,362
myname, dict_pgsql->parser->name, name);
return (0);
}
- #endif
/*
* Optionally fold the key.
--- 355,360
***
*** 659,685
/*
* The only legitimate encodings for Internet mail are ASCII and UTF-8.
*/
! #ifdef SNAPSHOT
! if (PQsetClientEncoding(host->db, "UTF8") != 0) {
! msg_warn("dict_pgsql: cannot set the encoding to UTF8, skipping %s",
!host->hostname);
! plpgsql_down_host(host);
! return;
}
- #else
/*
* XXX Postfix does not send multi-byte characters. The following piece
* of code is an explicit statement of this fact, and the database server
* should not accept multi-byte information after this point.
*/
! if (PQsetClientEncoding(host->db, "LATIN1") != 0) {
! msg_warn("dict_pgsql: cannot set the encoding to LATIN1, skipping %s",
!host->hostname);
! plpgsql_down_host(host);
! return;
}
- #endif
/* Success. */
host->stat = STATACTIVE;
}
--- 657,684
/*
* The only legitimate encodings for Internet mail are ASCII and UTF-8.
*/
! if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) != 0) {
! if (PQsetClientEncoding(host->db, "UTF8") != 0) {
! msg_warn("dict_pgsql: cannot set the encoding to UTF8, skipping %s",
!host->hostname);
! plpgsql_down_host(host);
! return;
! }
}
/*
* XXX Postfix does not send multi-byte characters. The following piece
* of code is an explicit statement of this fact, and the database server
* should not accept multi-byte information after this point.
*/
! else {
! if (PQsetClientEncoding(host->db, "LATIN1") != 0) {
! msg_warn("dict_pgsql: cannot set the encoding to LATIN1, skipping
%s",
!host->hostname);
! plpgsql_down_host(host);
! return;
! }
}
/* Success. */
host->stat = STATACTIVE;
}
Re: Patch: Encoding problem when talking to postgresql
On Tue, Feb 22, 2022 at 07:10:44PM -0500, Wietse Venema wrote:
> There are two #ifdef SNAPSHOT blocks.
>
> - The one in dict_pgsql_lookup() returns not found when SMTPUTF8
> is enabled, but a query is not valid UTF8.
>
> - The one in plpgsql_connect_single() sets the PgSQL client encoding
>to UTF8, and if SNAPSHOT is not defined, leaves it at LATIN1
>encoding which is what was used historically.
>
> How do we avoid unexpected breakages in a stable release?
>
> I suggest that we leave the encoding at LATIN1 when SMTPUTF8 is
> disabled. That is historical behavior and that minimizes the risk
> of breakages.
>
> We can then use the UTF8 encoding when SMTPUTF8 is enabled. It is
> the only encoding that makes sense with SMTPUTF8.
Sure, but there is some small risk that a site that enables UTF8
nevertheless is currently using a database with a server-side encoding
of LATIN1 and somehow avoiding actual use of UTF8 lookup keys.
This risk is probably quite modest. Introducing a dictionary
configuration keyword is fairly simple, with the proposed behaviour as a
default, but then also a mechanism to disable UTF8 if desired.
--- src/global/dict_pgsql.c
+++ src/global/dict_pgsql.c
@@ -207,6 +207,7 @@ typedef struct {
char *result_format;
void *ctx;
int expansion_limit;
+int utf8_enable;
char *username;
char *password;
char *dbname;
@@ -722,6 +723,7 @@ static void pgsql_parse_config(DICT_PGSQL *dict_pgsql,
const char *pgsqlcf)
dict_pgsql->password = cfg_get_str(p, "password", "", 0, 0);
dict_pgsql->dbname = cfg_get_str(p, "dbname", "", 1, 0);
dict_pgsql->result_format = cfg_get_str(p, "result_format", "%s", 1, 0);
+dict_pgsql->utf8_enable = cfg_get_bool(p, "utf8_enable", 1);
/*
* XXX: The default should be non-zero for safety, but that is not
The new dictionary parameter can then be used a guard to conditionally
suppress use of the UTF8 encoding in the PgSQL driver. But if that's a
bridge too far for a stable release, so be it.
(Of course the documentation would then have to describe the parater as
applicable from a certain patch level and up, IIRC there were some
precedents for this sort of emergency UI expansion, though none
recently).
--
Viktor.
Re: SASL hacking ?
On 20/02/22 05:35, Bill Cole wrote: We have listed all IPs. We can use a FW rule, but its heavy and hard to manage. A Postfix list may be easier. On Linux, using ipsets instead of putting IPs directly in rules helps a lot with managing large lists. Fail2ban can do its work via ipsets. An alternative to fail2ban and its workalikes is to more manually identify where your legitimate authentication traffic comes from and where it will never come from. Most of the IPs used for credential stuffing attacks are not associated with any end users or processes that would ever do any legitimate email submission anywhere, e.g. AWS instances, Digital Ocean Droplets, etc. You can safely refuse all port 465 and 587 traffic from huge swathes of the IPv4 space, if you are not a global mailbox provider. Whether you use fail2ban or do manual one-by-one blocking or use your logs to figure out broad boundaries between networks you'll allow to try to submit mail and ones you will not, the better place to do this is at the network layer, not Postfix. It is unlikely for a single credential stuffer to overwhelm your system, but they can cause resource problems in aggregate and that's much less likely if you're not even bothering to do a TCP session initiation. Geo-blocking is one strategy you might consider -- not just for Submission (e.g. port 587/465), but also for IMAP (e.g. port 143/993). There are sites which publish lists of IPv4 and IPv6 addresses for individual countries. The one I use is: https://www.ip2location.com/free/visitor-blocker Just remember that it will also stop you being able to access/send emails while on your overseas holiday! ;-) Nick.
Choosing relay based on sasl username?
Hi all,
I’m having some issues figuring out if this is possible and then if so, how to
approach it. This isn’t for production use, just some internal testing and
experimentation...
My goal is to set a relay destination based on the SASL username used for smtp
authentication. So if I have users “testsmtp1”, “testsmtp2”, “testsmtp3”, I’d
like to select the relay destination based on that username, NOT on the
envelope From.
I do have this version of it working, which does use the envelope From.
In main.cf:
# enable sasl auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_authenticated_header = yes
# map of sender to transport
sender_dependent_relayhost_maps = hash:/usr/local/etc/postfix/relay_by_sender
In the "relay_by_sender” map file:
testsmtp1 qmqprelay1
testsmtp2 qmqprelay2
testsmtp3 qmqprelay3
And in master.cf, the custom transports:
qmqprelay1 unix - n n - - pipe size=25
user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.9.8.200:631 ${sender}
${recipient}
qmqprelay2 unix - n n - - pipe size=25
user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.9.8.200:632 ${sender}
${recipient}
qmqprelay3 unix - n n - - pipe size=25
user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.99.8.200:633 ${sender}
${recipient}
In the above configuration, when I connect to relay using any of those
accounts, there is no match and the default transport is used.
If I add an entry to match the From address, that works fine, as expected.
So can I do this, but use the SASL username?
Thanks,
Charles
Re: Choosing relay based on sasl username?
Oops, sorry, "sender_dependent_relayhost_maps” should be
“sender_dependent_default_transport_maps” below...
> On Feb 23, 2022, at 2:17 AM, Charles Sprickman wrote:
>
> Hi all,
>
> I’m having some issues figuring out if this is possible and then if so, how
> to approach it. This isn’t for production use, just some internal testing and
> experimentation...
>
> My goal is to set a relay destination based on the SASL username used for
> smtp authentication. So if I have users “testsmtp1”, “testsmtp2”,
> “testsmtp3”, I’d like to select the relay destination based on that username,
> NOT on the envelope From.
>
> I do have this version of it working, which does use the envelope From.
>
> In main.cf:
>
> # enable sasl auth
> smtpd_sasl_auth_enable = yes
> smtpd_sasl_security_options = noanonymous
> smtpd_sasl_authenticated_header = yes
>
> # map of sender to transport
> sender_dependent_relayhost_maps = hash:/usr/local/etc/postfix/relay_by_sender
>
> In the "relay_by_sender” map file:
>
> testsmtp1 qmqprelay1
> testsmtp2 qmqprelay2
> testsmtp3 qmqprelay3
>
> And in master.cf, the custom transports:
>
> qmqprelay1 unix - n n - - pipe size=25
> user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.9.8.200:631 ${sender}
> ${recipient}
> qmqprelay2 unix - n n - - pipe size=25
> user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.9.8.200:632 ${sender}
> ${recipient}
> qmqprelay3 unix - n n - - pipe size=25
> user=qmqp argv=/usr/local/adm/bin/sendqmqp.pl 10.99.8.200:633 ${sender}
> ${recipient}
>
> In the above configuration, when I connect to relay using any of those
> accounts, there is no match and the default transport is used.
>
> If I add an entry to match the From address, that works fine, as expected.
>
> So can I do this, but use the SASL username?
>
> Thanks,
>
> Charles
