Fast forwarding Milters Re: Calling a SMTP Milter before local_recipient_maps
I apologize for my boundless optimism. I think the "it can't be done" mantra is really "it's always been this way". I understand that there is an impedance mismatch between the SMTP and milter protocols in that a milter is allowed to inspect and even reject an address supplied via SMTP RECV, but it is not allowed to edit it until the entire body has been processed (EOB/EOM). I see flags (SMFIP_NO*, SMFIP_NR_*) expressly intended to suppress large parts of the protocol exchange. Seems to me that for SMTP MAIL and RECV events at least, it is possible to use these flags to skip basically all of the intervening protocol goop between e.g. RECV and EOB/EOM (this works, PCAP of POC on request). I see no reason that a milter executor cannot fast forward to EOB/EOM based on these flags (seems to occur in milters when they're deciding what to ack or not); and that therefore milter edits to e.g. recipients in EOB/EOM can take place in the context of the SMTP RECV; and that a milter not desiring of this behavior can request intervening data / handshakes to disable this behavior, at least until an "allow_fast_forward" flag is decided upon (if ever). On 1/24/20 10:49 AM, Viktor Dukhovni wrote: > On Fri, Jan 24, 2020 at 10:04:26AM -0800, Fred Morris wrote: > >> I want to call a milter as a "bump in the wire" before this check to >> potentially alter local recipients prior to them ricocheting off of all >> of that shininess. >> >> net -> Postfix -> Postfix >> ^ ^ >> | | >> v v >> milter local_recip_maps > Milters get to process each SMTP command as it comes in, however > smtpd_recipient_restrictions are processed before handing off "RCPT TO" > to the milter. So the milter cannot modify the recipient address before > local_recipient_maps checks happen. > > You can use socketmap tables in local_recipient_maps if for some reason > you want to apply some non-trivial computation to the address in order > to determine whether it is valid. At least privately, the milter camp has provided aid and comfort. Whereas, nothing around domain socket support has risen above the usual background noise of people stating opinions about open source software they do not write or use; zero, nothing. Hence, milter! (At the moment I have no plans to write a milter executor and nobody has offered money, although I'd be happy to provide aid and comfort for anyone wanting to implement fast forwarding). Personally I'd rather disable the security checks preventing TCP tables in local(8) and recompile, but that's just my opinion. (I'll save my quibbles with Postfix's implementation of VRFY for another day.) -- Fred Morris
Re: Fast forwarding Milters Re: Calling a SMTP Milter before local_recipient_maps
Fred Morris: > I apologize for my boundless optimism. I think the "it can't be done" > mantra is really "it's always been this way". > > I understand that there is an impedance mismatch between the SMTP and > milter protocols in that a milter is allowed to inspect and even reject > an address supplied via SMTP RECV, but it is not allowed to edit it > until the entire body has been processed (EOB/EOM). You are describing something that is controlled by entities outside of Postfix. > I see flags (SMFIP_NO*, SMFIP_NR_*) expressly intended to suppress large > parts of the protocol exchange. Seems to me that for SMTP MAIL and RECV > events at least, it is possible to use these flags to skip basically all > of the intervening protocol goop between e.g. RECV and EOB/EOM (this > works, PCAP of POC on request). Always good to know this (still) works as promised. > I see no reason that a milter executor cannot fast forward to EOB/EOM > based on these flags (seems to occur in milters when they're deciding > what to ack or not); and that therefore milter edits to e.g. recipients > in EOB/EOM can take place in the context of the SMTP RECV; and that a The Milter protocol as defined supports envelope and content modifications after SMTP END-OF-DATA. In the case of Postfix, this means that the cleanup daemon makes the changes AFTER the entire envelope and message are stored in a queue file. > milter not desiring of this behavior can request intervening data / > handshakes to disable this behavior, at least until an > "allow_fast_forward" flag is decided upon (if ever). It requires a completely different Postfix implementation where the smtpd daemon receives the sender or recipient change request and makes the change in-memory, instead of the cleanup daemon receiving the change request and making the change in-file. Although technically possible, such a parallel implementation would be a very costly optimization for a really rare use case. Asking is cheap. Even explaining why it is a bad idea costs time. Wietse
Postfix can't find Postgserql's unix socket using "peer" method
Hello Using psql I can connect to postgresql's socket with a database user/role that does not have a password set. This is achieved by using "peer" authentication method and maps set in "pg_ident.conf" file: mailserver_map postfix mailserver "pg_hba.conf" file: localmailserverallpeer map=mailserver_map Issuing following command works and doesn't require password: sudo -u postfix psql -U mailserver -d mailserver -h /var/run/postgresql Additionally, all queries using "postmap -q" succeed with correct results. But none of the look ups defined in "virtual_mailbox_maps.cf" or " virtual_alias_maps.cf" work when the postfix server tries them, and in fact they report that connection to postgresql fails because it can't find it!: postfix/trivial-rewrite[8119]: warning: connect to pgsql server /var/run/postgresql: could not connect to server: No such file or directory??Is the server running locally and accepting??connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?? My virtual maps look like this: user = mailserver dbname = mailserver query = SELECT destination FROM virtual_aliases WHERE source = '%s' hosts = /var/run/postgresql My questions are: 1- Why Postfix reports that it can't connect to server instead of a message like "authentication failed for user"? Can this be considered a bug since it seems not providing "password" field breaks the command Postifx uses to establish communication to Postgres (hence the incorrect error message)? 2- Is there a way of achieving this authentication method that I am missing? I can make it work if I add a password for the database user and provide that in "virtual_alias_maps.cf" files but I was wondering if this can be done without password since I am using local/socket-based connection? Thanks in advance for your time, Hamid
Re: Postfix can't find Postgserql's unix socket using "peer" method
Dnia 28.01.2020 o godz. 18:43:48 Hamid M. pisze: > > Issuing following command works and doesn't require password: > sudo -u postfix psql -U mailserver -d mailserver -h /var/run/postgresql > > Additionally, all queries using "postmap -q" succeed with correct results. > > But none of the look ups defined in "virtual_mailbox_maps.cf" or " > virtual_alias_maps.cf" work when the postfix server tries them, and in fact > they report that connection to postgresql fails because it can't find it!: > > postfix/trivial-rewrite[8119]: warning: connect to pgsql server > /var/run/postgresql: could not connect to server: No such file or > directory??Is the server running locally and accepting??connections on Unix > domain socket "/var/run/postgresql/.s.PGSQL.5432"?? Maybe the reason is that your PostgreSQL socket isn't available from within Postfix chroot jail? Try placing it somewhere inside Postfix chroot jail. -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub."
Re: Postfix can't find Postgserql's unix socket using "peer" method
On Tue, Jan 28, 2020 at 6:52 PM Jaroslaw Rafa wrote: > > > postfix/trivial-rewrite[8119]: warning: connect to pgsql server > > /var/run/postgresql: could not connect to server: No such file or > > directory??Is the server running locally and accepting??connections on > Unix > > domain socket "/var/run/postgresql/.s.PGSQL.5432"?? > > Maybe the reason is that your PostgreSQL socket isn't available from within > Postfix chroot jail? > Try placing it somewhere inside Postfix chroot jail. > Thanks for your reply. I moved the socket folder into Postfix's jail "/var/spool/postfix/postgresql": ls -ld /var/spool/postfix/postgresql drwxrwsr-x 2 postgres postgres 4096 Jan 29 00:45 postgresql/ Postgresql starts correctly and creates the socket, however the issue remains, i.e, I can connect to sever using "psql" but I get same error messages from Postfix: Jan 29 00:47:19 mail postfix/trivial-rewrite[10046]: warning: connect to pgsql server /var/spool/postfix/postgresql: could not connect to server: No such file or directory?? Is the server running locally and accepting??connections on Unix domain socket "/var/spool/postfix/postgresql/.s.PGSQL.5432"??
Re: Postfix can't find Postgserql's unix socket using "peer" method
On Tue, Jan 28, 2020 at 07:53:42PM -0500, Hamid M. wrote: > Thanks for your reply. I moved the socket folder into Postfix's jail > "/var/spool/postfix/postgresql": It would be easier to just turn off chroot in master.cf, but: > ls -ld /var/spool/postfix/postgresql > drwxrwsr-x 2 postgres postgres 4096 Jan 29 00:45 postgresql/ You also need symbolic link /var/spool/postfix/var/spool/postfix -> ../.. So that paths under /var/spool/postfix work the same way for both chrooted and not chrooted processes. -- Viktor.
Re: Postfix can't find Postgserql's unix socket using "peer" method
On Tue, Jan 28, 2020 at 8:11 PM Viktor Dukhovni wrote: > > You also need symbolic link > > /var/spool/postfix/var/spool/postfix -> ../.. > > So that paths under /var/spool/postfix work the same way for > both chrooted and not chrooted processes. > > Nos sure how to go about doing this linking since I don't see any /var foledr wither /var/spool/postfix something like: cd /var/spool/postfix ln -s ../.. var/spool/postfix ?
Re: Postfix can't find Postgserql's unix socket using "peer" method
On Tue, Jan 28, 2020 at 08:27:53PM -0500, Hamid M. wrote: > > You also need symbolic link > > > > /var/spool/postfix/var/spool/postfix -> ../.. > > > > So that paths under /var/spool/postfix work the same way for > > both chrooted and not chrooted processes. > > Not sure how to go about doing this linking since I don't see any /var > folder wither /var/spool/postfix > > Something like: > cd /var/spool/postfix > ln -s ../.. var/spool/postfix Naturally, you'd have to create the intermediate directories: # (umask 022; mkdir -p /var/spool/postfix/var/spool) # ln -s ../.. /var/spool/postfix/var/spool/postfix -- Viktor.
Re: Postfix can't find Postgserql's unix socket using "peer" method
On Tue, Jan 28, 2020 at 8:36 PM Viktor Dukhovni wrote: > > Naturally, you'd have to create the intermediate directories: > > # (umask 022; mkdir -p /var/spool/postfix/var/spool) > # ln -s ../.. /var/spool/postfix/var/spool/postfix > Thanks Victor, this helped solve the issue. Just curious if I wanted to disable chroot, should it only apply to smtpd or some other few services?
Fond memories of 1988: OpenSMTPD remote code execution
If some of you have servers running OpenSMTPD, an update is in order:
https://www.openwall.com/lists/oss-security/2020/01/28/3
Dare I suggest Postfix, which has multiple layers of defense against
bugs of this sort.
The fact that recently developed software, reportedly designed with
security as a major goal, has a bug of this sort, is disappointing, but
also not that surprising:
* OpenSMTPD executes local delivery agents via a shell:
asprintf(&dispatcher->u.local.command,
"/usr/libexec/mail.local -f %%{mbox.from} %{user.username}");
execle("/bin/sh", "/bin/sh", "-c", mda_command, (char *)NULL,
mda_environ);
with the only barrier between safety and disaster a single sanity
check.
* OpenSMTPD is willing to deliver mail to root with root's privileges
* Perhaps more weakpoints, but those two are quite enough...
Thanks again Wietse, no sleepless nights fighting the next email worm
(or wondering who's been breaking into my MTA) for me...
--
Viktor.
