content_filter with external script and virtual_alias_maps

[Stats Student]

Hi, I would like to configure Postfix to do the following:

1) receive messages for users in a Postgres database and hand those
messages to an external script for processing (no traditional
mailstore). The server handles mail for only one domain.

2) for a handful of accounts (postmaster, help, root), the messages
should be forwarded to another address (different domain). Would be
great for this forwarding to take place without going through the
processing script in (1).

A very long time ago, I had it working with content_filter (as
described in http://www.postfix.org/FILTER_README.html#advanced_filter)
and virtual_alias_maps.
But for some reason (receive_override_options = no_address_mappings)
causes the server to ignore users in virtual_alias_maps and bounce.
Hence the question.

I used to use qpsmtpd on port 10025 with a custom plugin (in Perl)
which did the processing - but maybe there is a better way to do the
same.

What's the best way to get this configured? Would appreciate specific
setup recommendations (relay vs virtual aliases, content_filter, etc).
Happy to provide additional details.

Thanks.

Re: Virtual alias address class and no_address_mappings

[Peter]

On 30/12/19 8:42 pm, Viktor Dukhovni wrote:

With "ldap", "pgsql" and "mysql" it is generally a good idea to use
"proxy:ldap", "proxy:pgsql", ...


I agree, but in this particular case I was focusing on the problem at 
hand.  I find on IRC it pays to not always get caught up in every 
problem I see with someone's configuration if I want to see the end of them.



with "XFORWARD" the logs could be misleading, this could be the
processing of the message south of the content filter, and perhaps
the recipient is not listed in virtual_alias_maps.


While this is possible I doubt it because he did mention that removing 
no_address_mapping made the problem go away, so I think this is directly 
related to that setting.  Also he insisted that the address was in the 
db.  That said, it wouldn't be the first time I've seen someone say 
things like that on IRC that turned out not to be true.



 Is there any
other logging for "84E583E814" or the message-id in question?


None that was provided, but I did reference the OP to this thread so 
perhaps he will join in at this stage and provide more info.



If the "content_filter" was set as reported, the message would
not have been handed off to the error transport.  The presence
of a filter:

 
https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1105-L1115

preëmpts recipient-specific resolution:

 
https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1121-L1127

that (for example) bounces unknown recipients in virtual alias domains.

The reported symptoms are not consistent with content_filter being set
for the message.


Yes, but does it preempt the resolution of whether a recipient exists at 
all?  I thought that was done in smtpd, not qmgr.  I believe there might 
be an implicit check_recipient_access on the end of 
smtpd_recipient_restrictions that does that, but I'm not sure.  The idea 
being that a message needs to be rejected before it hits the queue?


That said the logs do show the message going straight into qmgr, so I'm 
really just baffled here.



Peter

Re: content_filter with external script and virtual_alias_maps

[Wietse Venema]

Stats Student:
> Hi, I would like to configure Postfix to do the following:
> 
> 1) receive messages for users in a Postgres database and hand those
> messages to an external script for processing (no traditional
> mailstore). The server handles mail for only one domain.

Postfix chooses the delivery method based on the recipient domain
(the part on the right-hand side of the '@'). 

Here, we direct mail for example.com to your message store, and we
make sure that all cronjob mail comes from r...@example.com,
w...@example.com etc.

/etc/postfix/main.cf
virtual_transport = mailstore
mailstore_destination_recipient_limit=1
virtual_mailbox_domains = example.com
virtual_mailbox_maps = hash:/etc/postfix/mailstore-users
myorigin = example.com
mydestination =

/etc/postfix/master.cf:
# Requires mailstore_destination_recipient_limit=1 in main.cf.
mailstore  unix  -   n   n   -   -   pipe
flags=DRhu user=vmail argv=/path/to/script ${recipient}

/etc/postfix/mailstore-users:
f...@example.comwhatever
b...@example.comwhatever

Once you have this working, replace hash:/etc/postfix/mailstore-users
with a pgsql table that returns a non-emnpty string for existing
users, and 'not found' otherwise. See "man pgsql_table", especially
the section "LIST MEMBERSHIP".

> 2) for a handful of accounts (postmaster, help, root), the messages
> should be forwarded to another address (different domain). Would be
> great for this forwarding to take place without going through the
> processing script in (1).

Use virtual_alias_maps:

/etc/postfix/main.cf:
virtual_alias_maps = hash:/etc/postfix/virtual

/etc/postfix/virtual:
postmaster  user1@example
rootuser2@example
helpuser3@example

No need to muck with content filters.

Wand ietse

Re: content_filter with external script and virtual_alias_maps

[Wietse Venema]

Wietse Venema:
> Stats Student:
> > Hi, I would like to configure Postfix to do the following:
> > 
> > 1) receive messages for users in a Postgres database and hand those
> > messages to an external script for processing (no traditional
> > mailstore). The server handles mail for only one domain.

With a few minor refinements:

Postfix chooses the delivery method based on the recipient domain
(the part on the right-hand side of the '@'). 

Here, we direct mail for example.com to your message store, and we
make sure that all cronjob mail comes from r...@example.com,
w...@example.com etc.

This uses the name 'mailstore' for all things related to your 
database. Of course you can use a different name.

/etc/postfix/main.cf
virtual_transport = mailstore
mailstore_destination_recipient_limit=1
virtual_mailbox_domains = example.com
virtual_mailbox_maps = hash:/etc/postfix/mailstore-users
myorigin = example.com
mydestination =

/etc/postfix/master.cf:
# Requires mailstore_destination_recipient_limit=1 in main.cf.
mailstore  unix  -   n   n   -   -   pipe
flags=DRXhu user=vmail argv=/path/to/script ${recipient}

/etc/postfix/mailstore-users:
f...@example.comwhatever
b...@example.comwhatever

This requires that you have have a UNIX system account 'vmail'
(command: useradd vmail). Of course you can use a different name.

Once you have this working, replace hash:/etc/postfix/mailstore-users
with a pgsql table that returns a non-emnpty string for existing
users, and 'not found' otherwise. See "man pgsql_table", especially
the section "LIST MEMBERSHIP".

> 2) for a handful of accounts (postmaster, help, root), the messages
> should be forwarded to another address (different domain). Would be
> great for this forwarding to take place without going through the
> processing script in (1).

Use virtual_alias_maps:

/etc/postfix/main.cf:
virtual_alias_maps = hash:/etc/postfix/virtual

/etc/postfix/virtual:
postmaster  user1@example
rootuser2@example
helpuser3@example

No need to muck with content filters.

Wietse

Re: Virtual alias address class and no_address_mappings

[Viktor Dukhovni]

On Mon, Dec 30, 2019 at 10:52:28PM +1300, Peter wrote:

> > The reported symptoms are not consistent with content_filter being set
> > for the message.
> 
> Yes, but does it preempt the resolution of whether a recipient exists at 
> all?  I thought that was done in smtpd, not qmgr.  I believe there might 
> be an implicit check_recipient_access on the end of 
> smtpd_recipient_restrictions that does that, but I'm not sure.

The existence checks in smtpd(8) are not affected by
receive_override_options.  The virtual alias tables are consulted as
appropriate.  In any case, the message was accepted.

In more sophisticated configurations, I've used different definitions of
virtual_alias_maps in smtpd(8) and cleanup(8), with the one used by
smtpd(8) used only for existence checks, avoiding expansion of LDAP
groups, ...  But since I wanted spam filtering (in particular a spam
quarantine) per-user, I did not use no_address_mappings.

> The idea being that a message needs to be rejected before it hits the
> queue?

This is why virtual_alias_maps is still used by smtpd(8) as part of
recipient validation.

> That said the logs do show the message going straight into qmgr, so I'm 
> really just baffled here.

The reported symptoms are not consistent with the use of a
content_filter.

-- 
Viktor.

Re: content_filter with external script and virtual_alias_maps

[Viktor Dukhovni]

On Mon, Dec 30, 2019 at 12:46:50AM -0800, Stats Student wrote:

> 1) receive messages for users in a Postgres database and hand those
> messages to an external script for processing (no traditional
> mailstore). The server handles mail for only one domain.
> 
> 2) for a handful of accounts (postmaster, help, root), the messages
> should be forwarded to another address (different domain).

This is easily handled via virtual_alias_maps, rewriting the recipients in
question.

> Would be great for this forwarding to take place without going through the
> processing script in (1).

Virtual alias rewriting happens before message delivery.  Note
that the domain need not be configured as a virtual alias domain,
virtual alias rewriting happens for all recipients, regardless
of address class.

> A very long time ago, I had it working with content_filter (as
> described in http://www.postfix.org/FILTER_README.html#advanced_filter)
> and virtual_alias_maps.

That's unnecessary, a pipe(8) transport is just fine, unless the mail is
going back into Postfix after processing by the script.

> But for some reason (receive_override_options = no_address_mappings)
> causes the server to ignore users in virtual_alias_maps and bounce.

Well, that's simply not how Postfix works, when you have a
content_filter.  The bounce only happens with no_address_mappings in
combination with NO content_filter.  But, you don't need either a filter
or no_address_mappings.

> What's the best way to get this configured? Would appreciate specific
> setup recommendations (relay vs virtual aliases, content_filter, etc).
> Happy to provide additional details.

Rewrite the exempt users to a different domain, resolve the original
domain to a pipe(8) transport that runs the script.  Provided the script
does not re-inject modified email back into Postfix with the same
recipient domain.

-- 
Viktor.

PROXY protocol v2 support

[Tamás Gérczei]

Hello List,

I'd like to ask if PROXY protocol v2 is supported by Postfix?

Thanks,
Tamás

Re: PROXY protocol v2 support

[Wietse Venema]

Tam?s G?rczei:
> Hello List,
> 
> I'd like to ask if PROXY protocol v2 is supported by Postfix?

It's not mentioned in documentation, therefore it is not supported.
Ditto for memcached v2 protocol.

Wietse

What are these types trying to do?

[Gerben Wierda]

Now that Finally have a postfix back with actual logging, I noticed this in my 
log:

Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
[182.99.42.88]:49546 to [192.168.2.66]:25
Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
[182.99.42.88]:49546: EHLO ylmf-pc\r\n
Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
unknown[182.99.42.88]
Dec 30 23:26:10 mail postfix/smtpd[16048]: disconnect from 
unknown[182.99.42.88] ehlo=1 commands=1
Dec 30 23:26:10 mail postfix/postscreen[16020]: CONNECT from 
[182.99.42.88]:49631 to [192.168.2.66]:25
Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.25 from 
[182.99.42.88]:49631: EHLO ylmf-pc\r\n
Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
Dec 30 23:26:11 mail postfix/smtpd[16048]: lost connection after EHLO from 
unknown[182.99.42.88]
Dec 30 23:26:11 mail postfix/smtpd[16048]: disconnect from 
unknown[182.99.42.88] ehlo=1 commands=1
Dec 30 23:26:14 mail postfix/postscreen[16020]: CONNECT from 
[182.99.42.88]:49966 to [192.168.2.66]:25
Dec 30 23:26:14 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
[182.99.42.88]:49966: EHLO ylmf-pc\r\n
Dec 30 23:26:14 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
Dec 30 23:26:14 mail postfix/smtpd[16048]: lost connection after EHLO from 
unknown[182.99.42.88]
Dec 30 23:26:14 mail postfix/smtpd[16048]: disconnect from 
unknown[182.99.42.88] ehlo=1 commands=1
Dec 30 23:26:18 mail postfix/postscreen[16020]: CONNECT from 
[182.99.42.88]:50289 to [192.168.2.66]:25
Dec 30 23:26:18 mail postfix/postscreen[16020]: PREGREET 14 after 0.25 from 
[182.99.42.88]:50289: EHLO ylmf-pc\r\n
Dec 30 23:26:18 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
Dec 30 23:26:18 mail postfix/smtpd[16048]: lost connection after EHLO from 
unknown[182.99.42.88]
Dec 30 23:26:18 mail postfix/smtpd[16048]: disconnect from 
unknown[182.99.42.88] ehlo=1 commands=1

And then lots of this. It goes on and on and on.

I was wondering (just curious) what these (Chinese) types are actually trying 
to do. It looks like polling based on the expectation that some other payload 
has corrupted my postfix. But I’m curious to what it really is (if known).

(Time to set a pf rule set on geolocation, I guess)

G

Re: What are these types trying to do?

[Viktor Dukhovni]

On Mon, Dec 30, 2019 at 11:32:11PM +0100, Gerben Wierda wrote:

> Now that Finally have a postfix back with actual logging, I noticed this in 
> my log:
> 
> Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
> [182.99.42.88]:49546 to [192.168.2.66]:25
> Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
> [182.99.42.88]:49546: EHLO ylmf-pc\r\n
> Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
> Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
> unknown[182.99.42.88]
> Dec 30 23:26:10 mail postfix/smtpd[16048]: disconnect from 
> unknown[182.99.42.88] ehlo=1 commands=1

Are the smtpd(8) connections on a different port?  One might expect
postscreen to block clients that send EHLO before the greeting is
received.

> And then lots of this. It goes on and on and on.

Welcome to the Internet...

> I was wondering (just curious) what these (Chinese) types are actually
> trying to do. It looks like polling based on the expectation that some
> other payload has corrupted my postfix. But I’m curious to what it
> really is (if known).

It doesn't matter.

> (Time to set a pf rule set on geolocation, I guess)

I wouldn't bother, but since the host has no PTR record you can,
just in case, add:

reject_unknown_reverse_client_hostname

to your smtpd_client_restrictions.

-- 
Viktor.

Re: What are these types trying to do?

[Allen Coates]

On 30/12/2019 22:32, Gerben Wierda wrote:
> Now that Finally have a postfix back with actual logging, I noticed this in 
> my log:
> 
> Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
> [182.99.42.88]:49546 to [192.168.2.66]:25
> Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
> [182.99.42.88]:49546: EHLO ylmf-pc\r\n
> Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
> Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
> unknown[182.99.42.88]

<< etc >>

if you set the parameter

postscreen_greet_action = ENFORCE, or
postscreen_greet_action = DROP

these connections will be held back by postscreen, and will not actually reach
postfix.

The ENFORCE option will collect the (envelope) FROM and TO addresses for stats
purposes, if they are proffered.

Hope this helps

Allen C

Re: What are these types trying to do?

[Benny Pedersen]

Viktor Dukhovni skrev den 2019-12-30 23:46:

Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
[182.99.42.88]:49546 to [192.168.2.66]:25
Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 
from [182.99.42.88]:49546: EHLO ylmf-pc\r\n


https://blog.sys4.de/abwehr-des-botnets-pushdo-cutwail-ehlo-ylmf-pc-mit-iptables-string-recent-smtp-de.html

to remove noice in log files

# cat shorewall-rules
?SECTION ESTABLISHED
DROP net $FW tcp 25;;-m string --algo bm --string "EHLO ylmf-pc"

Re: What are these types trying to do?

[Wietse Venema]

Gerben Wierda:
> Now that Finally have a postfix back with actual logging, I noticed this in 
> my log:
> 
> Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
> [182.99.42.88]:49546 to [192.168.2.66]:25
> Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
> [182.99.42.88]:49546: EHLO ylmf-pc\r\n
> Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
> Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
> unknown[182.99.42.88]
> Dec 30 23:26:10 mail postfix/smtpd[16048]: disconnect from 
> unknown[182.99.42.88] ehlo=1 commands=1

This a very common spambot. Postfix sends

220-$smtpd_banner

and it talks before its turn with:

 EHLO ylmf-pc

These bots are very stupid and very persistent. My maillog file for
today has 3500 of these, and that is with 6 more hours to go.

> I was wondering (just curious) what these (Chinese) types are
> actually trying to do.

Trying to send spam, with a borked SMTP implementation. This is
the most common postscreen pregreet pattern.

> It looks like polling based on the expectation that some other
> payload has corrupted my postfix. But I?m curious to what it really
> is (if known).

You are vastly overestimating this spambot.

Wietse

Re: What are these types trying to do?

[Gerben Wierda]

> On 30 Dec 2019, at 23:46, Viktor Dukhovni  > wrote:
> 
> On Mon, Dec 30, 2019 at 11:32:11PM +0100, Gerben Wierda wrote:
> 
>> Now that Finally have a postfix back with actual logging, I noticed this in 
>> my log:
>> 
>> Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
>> [182.99.42.88]:49546 to [192.168.2.66]:25
>> Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
>> [182.99.42.88]:49546: EHLO ylmf-pc\r\n
>> Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
>> Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
>> unknown[182.99.42.88]
>> Dec 30 23:26:10 mail postfix/smtpd[16048]: disconnect from 
>> unknown[182.99.42.88] ehlo=1 commands=1
> 
> Are the smtpd(8) connections on a different port?  One might expect
> postscreen to block clients that send EHLO before the greeting is
> received.
> 

# SMTP on port 25, handled by postscreen before passing it on to
# smtpd
smtp  inet  n   -   n   -   1   postscreen
# smtpd, which listens on /opt/local/var/spool/postfix/smtpd
smtpd pass  -   -   n   -   -   smtpd
#  -o receive_override_options=no_address_mappings
# dnsblog, which listens on /opt/local/var/spool/postfix/dnsblog
dnsblog   unix  -   -   n   -   0   dnsblog
# tlsproxy, which listens on /opt/local/var/spool/postfix/tlsproxy
tlsproxy  unix  -   -   n   -   0   tlsproxy
# submission which listens on port 587:
# TLS and authentication are required on this port
submission inet n   -   n   -   -   smtpd
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o syslog_name=submission


So, postscreen gets the smtp connection (port 25). smtpd listens directly on 
the submission port. (I have been wondering to use 465 again, since IANA has 
reassigned it for secure SMTP).

>> And then lots of this. It goes on and on and on.
> 
> Welcome to the Internet…

Since 1990-1991 ;-)

I know.

> 
>> I was wondering (just curious) what these (Chinese) types are actually
>> trying to do. It looks like polling based on the expectation that some
>> other payload has corrupted my postfix. But I’m curious to what it
>> really is (if known).
> 
> It doesn't matter.
> 
>> (Time to set a pf rule set on geolocation, I guess)
> 
> I wouldn't bother, but since the host has no PTR record you can,
> just in case, add:
> 
>reject_unknown_reverse_client_hostname
> 
> to your smtpd_client_restrictions.

Yes. Hmm, does that come with a big risk for stopping legitimate mail? Probably 
yes given the amount of poorly written smtp clients on web sites.

> 
> -- 
>Viktor.

Re: What are these types trying to do?

[Wietse Venema]

Wietse Venema:
> Gerben Wierda:
> > Now that Finally have a postfix back with actual logging, I noticed this in 
> > my log:
> > 
> > Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
> > [182.99.42.88]:49546 to [192.168.2.66]:25
> > Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
> > [182.99.42.88]:49546: EHLO ylmf-pc\r\n
> > Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from 
> > unknown[182.99.42.88]
> > Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
> > unknown[182.99.42.88]
> > Dec 30 23:26:10 mail postfix/smtpd[16048]: disconnect from 
> > unknown[182.99.42.88] ehlo=1 commands=1
> 
> This a very common spambot. Postfix sends
> 
>   220-$smtpd_banner
> 
> and it talks before its turn with:
> 
>EHLO ylmf-pc
> 
> These bots are very stupid and very persistent. My maillog file for
> today has 3500 of these, and that is with 6 more hours to go.

Oh, and I do ENFORCE the pregreet test, so these bots never get
to talk to a Postfix SMTP daemon. 

Wietse

> > I was wondering (just curious) what these (Chinese) types are
> > actually trying to do.
> 
> Trying to send spam, with a borked SMTP implementation. This is
> the most common postscreen pregreet pattern.
> 
> > It looks like polling based on the expectation that some other
> > payload has corrupted my postfix. But I?m curious to what it really
> > is (if known).
> 
> You are vastly overestimating this spambot.
> 
>   Wietse
> 

Re: What are these types trying to do?

[Viktor Dukhovni]

On Tue, Dec 31, 2019 at 12:20:58AM +0100, Gerben Wierda wrote:

> > since the host has no PTR record you can, just in case, add:
> > 
> >reject_unknown_reverse_client_hostname
> > 
> > to your smtpd_client_restrictions.
> 
> Yes. Hmm, does that come with a big risk for stopping legitimate mail?
> Probably yes given the amount of poorly written smtp clients on web
> sites.

Actually, requiring remote SMTP clients to have a PTR record is
reasonably safe.  The less safe version, that is not recommended, is
requiring that PTR record to also forward resolve to the connecting IP
address.

Of course you could be unlucky enough to receive important email from
such clients, but that is not at all common, and such senders would
already be blocked at most SMTP servers on the public Internet.

-- 
Viktor.

Re: What are these types trying to do?

[Gerben Wierda]

> On 31 Dec 2019, at 00:11, Allen Coates  wrote:
> 
> 
> 
> On 30/12/2019 22:32, Gerben Wierda wrote:
>> Now that Finally have a postfix back with actual logging, I noticed this in 
>> my log:
>> 
>> Dec 30 23:26:09 mail postfix/postscreen[16020]: CONNECT from 
>> [182.99.42.88]:49546 to [192.168.2.66]:25
>> Dec 30 23:26:10 mail postfix/postscreen[16020]: PREGREET 14 after 0.26 from 
>> [182.99.42.88]:49546: EHLO ylmf-pc\r\n
>> Dec 30 23:26:10 mail postfix/smtpd[16048]: connect from unknown[182.99.42.88]
>> Dec 30 23:26:10 mail postfix/smtpd[16048]: lost connection after EHLO from 
>> unknown[182.99.42.88]
> 
> << etc >>
> 
> if you set the parameter
> 
> postscreen_greet_action = ENFORCE, or

I’ve done this as well as the  reject_unknown_reverse_client_hostname

G

How to get successful delivery reported in miy log?

[Gerben Wierda]

Trying to get my postfix logging fully in order.

When a mail is sent to the outside world, my log shows:

Dec 31 01:21:32 mail submission/smtpd[16982]: connect from 
luna-wifi.rna.nl[192.168.2.89]
Dec 31 01:21:33 mail submission/smtpd[16982]: 06C9C19C940A: 
client=luna-wifi.rna.nl[192.168.2.89], sasl_method=CRAM-MD5, 
sasl_username=[snip]
Dec 31 01:21:33 mail postfix/cleanup[16985]: 06C9C19C940A: 
message-id=
Dec 31 01:21:33 mail postfix/qmgr[16777]: 06C9C19C940A: 
from=, size=2615, nrcpt=1 (queue active)
Dec 31 01:21:34 mail postfix/qmgr[16777]: 06C9C19C940A: removed

I don’t see logging that it was successfully handed to another smtp-server. 
I’ve tried adding -v to the smtpd commands in master.cf but that doesn’t really 
help.

I can’t use syslog on my system so I’m using postlog.

Gerben Wierda
Chess and the Art of Enterprise Architecture 
Mastering ArchiMate 
Architecture for Real Enterprises 
 at InfoWorld
On Slippery Ice  at EAPJ

Re: What are these types trying to do?

[Gerben Wierda]

> On 31 Dec 2019, at 00:24, Wietse Venema  wrote:
> 
>> These bots are very stupid and very persistent. My maillog file for
>> today has 3500 of these, and that is with 6 more hours to go.
> 

9500 in 13 hours here. With the new settings (ENFORCE) smtpd is spared but I 
still have this junk in my log

Definitely going to look into pf and blocking geolocations.

G

Re: How to get successful delivery reported in miy log?

[Wietse Venema]

Gerben Wierda:
> Trying to get my postfix logging fully in order.
> 
> When a mail is sent to the outside world, my log shows:
> 
> Dec 31 01:21:32 mail submission/smtpd[16982]: connect from 
> luna-wifi.rna.nl[192.168.2.89]
> Dec 31 01:21:33 mail submission/smtpd[16982]: 06C9C19C940A: 
> client=luna-wifi.rna.nl[192.168.2.89], sasl_method=CRAM-MD5, 
> sasl_username=[snip]
> Dec 31 01:21:33 mail postfix/cleanup[16985]: 06C9C19C940A: 
> message-id=
> Dec 31 01:21:33 mail postfix/qmgr[16777]: 06C9C19C940A: 
> from=, size=2615, nrcpt=1 (queue active)
> Dec 31 01:21:34 mail postfix/qmgr[16777]: 06C9C19C940A: removed
> 
> I don?t see logging that it was successfully handed to another smtp-server. 
> I?ve tried adding -v to the smtpd commands in master.cf but that doesn?t 
> really help.

I remember that you reported a bug where a program cant talk to
postlogd if it opens the postlog socked after dropping privileges.

I posted a patch for that, but I never heard back if that worked,
and therefore that patch is not part of Postfix source code.

Wietse

diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL 
--exclude=.indent.pro --exclude=Makefile.in -r -ur 
/var/tmp/postfix-3.5-20190922/HISTORY ./HISTORY
--- /var/tmp/postfix-3.5-20190922/HISTORY   2019-09-21 11:54:59.0 
-0400
+++ ./HISTORY   2019-10-12 16:32:04.0 -0400
@@ -24411,3 +24411,9 @@
calls. This allows tlsproxy(8) to reset an I/O timer after
each event without having to make an nbbio_disable_readwrite()
call. Files: util/nbbio.c, tlsproxy/tlsproxy.c.
+
+20191007
+
+   Workaround: postlog clients open the socket before entering
+   the chroot jail and before dropping privileges. Files:
+   util/msg_logger.[hc], global/maillog_client.c.
diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL 
--exclude=.indent.pro --exclude=Makefile.in -r -ur 
/var/tmp/postfix-3.5-20190922/src/global/maillog_client.c 
./src/global/maillog_client.c
--- /var/tmp/postfix-3.5-20190922/src/global/maillog_client.c   2019-01-30 
19:41:59.0 -0500
+++ ./src/global/maillog_client.c   2019-10-07 19:14:33.0 -0400
@@ -210,7 +210,7 @@
if (var_maillog_file && *var_maillog_file) {
ARGV   *good_prefixes = argv_split(var_maillog_file_pfxs,
   CHARS_COMMA_SP);
-   char **cpp;
+   char  **cpp;
 
for (cpp = good_prefixes->argv; /* see below */ ; cpp++) {
if (*cpp == 0)
@@ -264,6 +264,8 @@
}
if (service_path != import_service_path)
myfree(service_path);
+   msg_logger_control(CA_MSG_LOGGER_CTL_CONNECT_NOW,
+  CA_MSG_LOGGER_CTL_END);
 }
 
 /*
diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL 
--exclude=.indent.pro --exclude=Makefile.in -r -ur 
/var/tmp/postfix-3.5-20190922/src/util/msg_logger.c ./src/util/msg_logger.c
--- /var/tmp/postfix-3.5-20190922/src/util/msg_logger.c 2019-01-29 
17:24:42.0 -0500
+++ ./src/util/msg_logger.c 2019-10-12 16:27:46.0 -0400
@@ -62,6 +62,10 @@
 /* .IP CA_MSG_LOGGER_CTL_DISABLE
 /* Disable the msg_logger. This remains in effect until the
 /* next msg_logger_init() call.
+/* .IP CA_MSG_LOGGER_CTL_CONNECT_NOW
+/* Close the logging socket if it was already open, and open
+/* the logging socket now, if permitted by current settings.
+/* Otherwise, the open is delayed until a logging request.
 /* SEE ALSO
 /* msg(3)  diagnostics module
 /* BUGS
@@ -111,6 +115,8 @@
 static int msg_logger_fallback_only_override = 0;
 static int msg_logger_enable = 0;
 
+#define MSG_LOGGER_NEED_SOCKET() (msg_logger_fallback_only_override == 0)
+
  /*
   * Other state.
   */
@@ -130,6 +136,26 @@
 #define STR(x) vstring_str(x)
 #define LEN(x) VSTRING_LEN(x)
 
+/* msg_logger_connect - connect to logger service */
+
+static void msg_logger_connect(void)
+{
+if (msg_logger_sock == MSG_LOGGER_SOCK_NONE) {
+   msg_logger_sock = unix_dgram_connect(msg_logger_unix_path, BLOCKING);
+   if (msg_logger_sock >= 0)
+   close_on_exec(msg_logger_sock, CLOSE_ON_EXEC);
+}
+}
+
+/* msg_logger_disconnect - disconnect from logger service */
+
+static void msg_logger_disconnect(void)
+{
+if (msg_logger_sock != MSG_LOGGER_SOCK_NONE) {
+   (void) close(msg_logger_sock);
+   msg_logger_sock = MSG_LOGGER_SOCK_NONE;
+}
+}
 
 /* msg_logger_print - log info to service or file */
 
@@ -203,12 +229,8 @@
  * will report ENOENT if the endpoint does not exist, ECONNREFUSED if no
  * server has opened the endpoint.
  */
-if (msg_logger_fallback_only_override == 0
-   && msg_logger_sock == MSG_LOGGER_SOCK_NONE) {
-   msg_logger_sock = unix_dgram_connect(msg_logger_unix_path, BLOCKING);
-   if (msg_logger_sock >= 0)
-   close_on_exec(msg_logger_sock, CLOSE_ON_EXEC);
-}
+if (MSG_LOGGER_N

Re: How to get successful delivery reported in miy log?

[Gerben Wierda]

> On 31 Dec 2019, at 01:57, Wietse Venema  wrote:
> 
> I remember that you reported a bug where a program cant talk to
> postlogd if it opens the postlog socked after dropping privileges.
> 
> I posted a patch for that, but I never heard back if that worked,
> and therefore that patch is not part of Postfix source code.
> 

I see. I wasn’t aware I was supposed to test. I am using the postfix 
distribution via macports and have only done light adaptations (of Portfiles), 
not yet including patching. I am also not maintainer of this port.

I’ll see if I can have a look.

G

Re: Virtual alias address class and no_address_mappings

[Stats Student]

Thanks for looking into this and thank you Peter for reporting my
issue to the list.

To confirm, the alias is in the last database (
hash:/etc/postfix/virtual_alias_maps ) but the forwarding only works
if "receive_override_options = no_address_mappings" is NOT set
(commented out). And for the forwarding to actually happen the
content_filter must be disabled (unless there is a valid daemon
listening and re-injecting, etc). If no_address_mappings is enabled,
the message bounces with user unknown error.

virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf
hash:/etc/postfix/virtual_alias_maps

Re: content_filter -- yes, I did uncomment this setting when
submitting the configuration to provide a fuller picture of my setup.
However, as you noted, the log entries were generated without the
content_filter set. Apologies for the confusion.

On Mon, Dec 30, 2019 at 8:51 AM Viktor Dukhovni
 wrote:
>
> On Mon, Dec 30, 2019 at 10:52:28PM +1300, Peter wrote:
>
> > > The reported symptoms are not consistent with content_filter being set
> > > for the message.
> >
> > Yes, but does it preempt the resolution of whether a recipient exists at
> > all?  I thought that was done in smtpd, not qmgr.  I believe there might
> > be an implicit check_recipient_access on the end of
> > smtpd_recipient_restrictions that does that, but I'm not sure.
>
> The existence checks in smtpd(8) are not affected by
> receive_override_options.  The virtual alias tables are consulted as
> appropriate.  In any case, the message was accepted.
>
> In more sophisticated configurations, I've used different definitions of
> virtual_alias_maps in smtpd(8) and cleanup(8), with the one used by
> smtpd(8) used only for existence checks, avoiding expansion of LDAP
> groups, ...  But since I wanted spam filtering (in particular a spam
> quarantine) per-user, I did not use no_address_mappings.
>
> > The idea being that a message needs to be rejected before it hits the
> > queue?
>
> This is why virtual_alias_maps is still used by smtpd(8) as part of
> recipient validation.
>
> > That said the logs do show the message going straight into qmgr, so I'm
> > really just baffled here.
>
> The reported symptoms are not consistent with the use of a
> content_filter.
>
> --
> Viktor.

Broken Resource Links

[Peter]

Unfortunately it appears that Patrik Koetter's website is now defunct 
and the domain has been taken over by someone else completely leaving a 
number of resources linked to from postfix.org broken including (but 
probably not limited to):


The saslfinger utility linked to from DEBUG_README.

The Postfix SASL Authentication and TLS howto linked to in two different 
places on the docs page.


...possibly others.

Fortunately There are copies of the above listed resources on 
Archive.org's wayback machine:


https://web.archive.org/web/20190327234908/http://postfix.state-of-mind.de/patrick.koetter/saslfinger/

https://web.archive.org/web/20190618125312/http://postfix.state-of-mind.de/patrick.koetter/smtpauth/

It might be useful if someone wants to copy those resources to a more 
permanent location, or at least update the links.



Peter

Re: Virtual alias address class and no_address_mappings

[Viktor Dukhovni]

On Mon, Dec 30, 2019 at 06:18:34PM -0800, Stats Student wrote:

> Re: content_filter -- yes, I did uncomment this setting when
> submitting the configuration to provide a fuller picture of my setup.
> However, as you noted, the log entries were generated without the
> content_filter set. Apologies for the confusion.

And of course "receive_override_options" MUST NOT be used when you don't
have a content_filter.  It is intended to make it possible to delay some
checks to post-filter processing, and exclude others already performed,
but absent a content_filter, it just breaks things by disabling required
some processing.

Please NEVER report configuration settings that are different from those
that were in effect when the reported logs were generated.  That wastes
the time of people trying to help you, and is simply rude, however well
intentioned (that "fuller picture" was anything but).

-- 
Viktor.

Re: What are these types trying to do?

[Viktor Dukhovni]

On Tue, Dec 31, 2019 at 01:50:43AM +0100, Gerben Wierda wrote:

> 9500 in 13 hours here. With the new settings (ENFORCE) smtpd is spared but I
> still have this junk in my log
> 
> Definitely going to look into pf and blocking geolocations.

Accumulation of a pile of ad-hoc filter rules makes your MTA brittle
over time.  My advice is to just ignore the noise in the logs, there'll
always be new noise.  Just let it go.  Postscreen is working for you,
without custom rules for each new source.

-- 
Viktor.

Re: PROXY protocol v2 support

[Tamás Gérczei]

Thanks Wietse, this is what I thought and found out during my
experiments,That said, now knowing that only v1 is supported, may I ask
whether you have considered implementing v2 support? I'm about to
migrate to a setup where I'm behind a load balancer that only speaks v2.

Yours,
Tamás

On 12/30/19 9:38 PM, Wietse Venema wrote:
> Tam?s G?rczei:
>> Hello List,
>>
>> I'd like to ask if PROXY protocol v2 is supported by Postfix?
> It's not mentioned in documentation, therefore it is not supported.
> Ditto for memcached v2 protocol.
>
>   Wietse

Re: PROXY protocol v2 support

[Willy Tarreau]

On Tue, Dec 31, 2019 at 08:21:05AM +0100, Tamás Gérczei wrote:
> Thanks Wietse, this is what I thought and found out during my
> experiments,That said, now knowing that only v1 is supported, may I ask
> whether you have considered implementing v2 support? I'm about to
> migrate to a setup where I'm behind a load balancer that only speaks v2.

Maybe you can try to implement v2 support ? Parsing v2 when v1 is already
supported is quite easy, especially at the same level of support (i.e. no
TLV field support for TLS or whatever). You can have a look at
conn_recv_proxy() in haproxy:src/connection.c which supports the two
versions. Feel free to steal any code I wrote there, if that helps :-)

Regards,
Willy