Re: Limiting global number of outgoing connections

[Wietse Venema]

Viktor Dukhovni:
> Likely I am missing some key insight, please pardon any confusion...

The ISP provides a budget of 5 connections/second, enforced at the
firewall level. The OP's logging shows ECONNREFUSED when the Postfix
scheduler rate limits to <1 delivery request per second. How can
that be?

For one delivery request, the Postfix SMTP client gets up to
five ($smtp_mx_address_limit) addresses from DNS, and burns through
those addresses in a split second as they fail with ECONNREFUSED.
Instantly, he has used up his entire 5 connections/second budget
with only one delivert request.

Obviously, the queue scheduler has no idea how many 'connections'
are being used up per delivery request. No amount of scheduler
tweaking will change that.

Wietse

policy server, TLS only exeptions and restrictions

[Stefan Bauer]

Hi,

we have enforced TLS to all remote sites and have appropriate tls policy
server, that checks if TLS is avail before accepting mails. That works as
expected. we also only accept users with auth.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated
reject_unauth_destination

smtpd_recipient_restrictions = check_policy_service unix:private/policy

policy server returns dunno or defer...

Now the problem:

for some destinations, we are aware, that TLS fails, so we skip checking
and set "may" policy for specific users/destinations. However this settings
seems to have no effect anymore, when we enable check_policy_service.

master.cf (snippet):
finance  unix -   -   n   -   -   smtp
smtp_tls_policy_maps=hash:/etc/postfix/tls/finance

tls/finance:
remote-site.de may

policy server responds with defer and custom smtp_tls_policy_maps are
ignored.

Howto work around this?

thank you.

Stefan

Re: policy server, TLS only exeptions and restrictions

[Matus UHLAR - fantomas]

On 04.01.19 14:44, Stefan Bauer wrote:

we have enforced TLS to all remote sites and have appropriate tls policy
server, that checks if TLS is avail before accepting mails. That works as
expected. we also only accept users with auth.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated
reject_unauth_destination

smtpd_recipient_restrictions = check_policy_service unix:private/policy

policy server returns dunno or defer...

Now the problem:

for some destinations, we are aware, that TLS fails, so we skip checking
and set "may" policy for specific users/destinations. However this settings
seems to have no effect anymore, when we enable check_policy_service.

master.cf (snippet):
finance  unix -   -   n   -   -   smtp
smtp_tls_policy_maps=hash:/etc/postfix/tls/finance

tls/finance:
remote-site.de may

policy server responds with defer and custom smtp_tls_policy_maps are
ignored.

Howto work around this?


this looks to me that you search for connection between 
smtpd_recipient_restrictions
and smtp_tls_policy_maps, and there is none.

the "check_policy_service private/policy" communicates via unix socket
private/policy (apparetly in postfix directory) to external program that
tells smtpd what to do.

if you want your policy server to return dunno for sending domain
"remote-site.de", your policy server must look to the /etc/postfix/tls/finance
table for the remote-site.de domain.

the policy server doesn't look to your "smtp_tls_policy_maps" settings,
usually it does not read postfix configuration at all.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Nothing is fool-proof to a talented fool. 

Slowness after upgrading from postfix 2.x to 3.1.8

[Christopher R. Gabriel]

Hi,

after upgrading to Debian 9 (thus Postfix 3.1.8) I'm experiecing an odd
behaviour, which causes slowness on all the infrastructure.

I have a generator server which injects (via smtp) into postfix, the
actual sender, and when burst of delivery happens, the receiving
postfix stuck before answering to the generator, which causes the
generator queues to fill up.

I've logged the smtp session under load from the postfix 3.1.8 server,
and in the following log excerpt you can note a "pause" of ~26 seconds:

Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 9 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 7 data   
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 36 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 67 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 0 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 55 data --=-WW
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type N
len 0 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type X
len 0 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type E
len 0 data 
Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]:
vstream_fflush_some: fd 18 flush 2433
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]:
vstream_buf_get_ready: fd 18 got 18
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: public/cleanup
socket: wanted attribute: status
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: input attribute
name: status
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: input attribute
value: 0
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: public/cleanup
socket: wanted attribute: reason
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: input attribute
name: reason
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: input attribute
value: (end)
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: public/cleanup
socket: wanted attribute: (list terminator)
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: input attribute
name: (end)
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: >
gen01.localdomain[192.168.25.135]: 250 2.0.0 Ok: queued as
ECA962EA0B336
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]:
vstream_fflush_some: fd 10 flush 39
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: abort all milters
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: milter8_abort:
abort milter inet:127.0.0.1:12301
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: watchdog_pat:
0x561d4886f850
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]:
vstream_buf_get_ready: fd 10 got 6
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: <
gen01.localdomain[192.168.25.135]: QUIT
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: >
gen01.localdomain[192.168.25.135]: 221 2.0.0 Bye
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: match_hostname:
smtpd_client_event_limit_exceptions: gen01.localdomain ~? 127.0.0.0/8

I can provide the full session log.

postfix01 data/spool are on tmpfs.

Do you have any suggestions? This is unfortunately stopping the upgrade
to postfix 3.x for the whole infrastructure.

Thank for any help you can provide!

Regards,

Christopher


alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
bounce_queue_lifetime = 6h
compatibility_level = 2
config_directory = /etc/postfix
connection_cache_ttl_limit = 16s
data_directory = /var/spool/postfix/data
default_destination_concurrency_limit = 200
default_destination_rate_delay = 0s
default_destination_recipient_limit = 50
default_recipient_limit = 40
default_transport = defer
disable_mime_output_conversion = yes
header_checks =  regexp:/etc/postfix/header_checks
in_flow_delay = 1
inet_interfaces = 192.168.25.131, 127.0.0.1
inet_protocols = ipv4
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
maximal_backoff_time = 4000s
maximal_queue_lifetime = 3h
milter_default_action = accept
milter_protocol = 6
minimal_backoff_time = 300s
mydestination =
myhostname = postfix01.localdomain
mynetworks = $config_directory/mynetworks
myorigin = /etc/mailname
non_smtpd_milters = $smtpd_milters
qmgr_message_active_limit = 8
qmgr_message_recipient_limit = 8
queue_directory = /var/spool/postfix
readme_directory = no
recipient_delimiter = +
relayhost =
smtp_connect_timeout = 3
smtp_connection_cache_on_demand = no
smtp_connection_cache_time_limit = 16s
smtp_connection_reuse_time_limit = 16s
smtp_discard_ehlo_keyword_address_maps = hash:/etc/postfix/busted-
servers
smtp_helo_timeout = 30
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtp_tls_enforce_peername = no
smtp_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtp_tls_loglevel = 0
smtp_tls_mandatory_ciphers = low
smtp_tls_security_level = may
smtp_tls_session_cache_timeout = 0
smt

Re: policy server, TLS only exeptions and restrictions

[Stefan Bauer]

Understood. Thank you.

Am Fr., 4. Jan. 2019 um 15:11 Uhr schrieb Matus UHLAR - fantomas <
uh...@fantomas.sk>:

> On 04.01.19 14:44, Stefan Bauer wrote:
> >we have enforced TLS to all remote sites and have appropriate tls policy
> >server, that checks if TLS is avail before accepting mails. That works as
> >expected. we also only accept users with auth.
> >
> >smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated
> >reject_unauth_destination
> >
> >smtpd_recipient_restrictions = check_policy_service unix:private/policy
> >
> >policy server returns dunno or defer...
> >
> >Now the problem:
> >
> >for some destinations, we are aware, that TLS fails, so we skip checking
> >and set "may" policy for specific users/destinations. However this
> settings
> >seems to have no effect anymore, when we enable check_policy_service.
> >
> >master.cf (snippet):
> >finance  unix -   -   n   -   -   smtp
> >smtp_tls_policy_maps=hash:/etc/postfix/tls/finance
> >
> >tls/finance:
> >remote-site.de may
> >
> >policy server responds with defer and custom smtp_tls_policy_maps are
> >ignored.
> >
> >Howto work around this?
>
> this looks to me that you search for connection between
> smtpd_recipient_restrictions
> and smtp_tls_policy_maps, and there is none.
>
> the "check_policy_service private/policy" communicates via unix socket
> private/policy (apparetly in postfix directory) to external program that
> tells smtpd what to do.
>
> if you want your policy server to return dunno for sending domain
> "remote-site.de", your policy server must look to the
> /etc/postfix/tls/finance
> table for the remote-site.de domain.
>
> the policy server doesn't look to your "smtp_tls_policy_maps" settings,
> usually it does not read postfix configuration at all.
>
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Nothing is fool-proof to a talented fool.
>

bypass policy server in recipient_restrictions when subject contains string

[Stefan Bauer]

Hi,

is there a way to bypass policy server in smtp_recipient_restrictions, in
case, subject contains special string?

smtpd_recipient_restrictions = check_policy_service unix:private/policy

header_checks:

/^Subject: .*string.*/ FILTER no-policy-service:

header_checks could reroute by subject but seems to kick in too late :/

Stefan

Re: Slowness after upgrading from postfix 2.x to 3.1.8

[Viktor Dukhovni]

> On Jan 4, 2019, at 9:23 AM, Christopher R. Gabriel 
>  wrote:
> 
> Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type E
> len 0 data 
> Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]:
> vstream_fflush_some: fd 18 flush 2433
> Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]:
> vstream_buf_get_ready: fd 18 got 18
> Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: public/cleanup
> socket: wanted attribute: status

Your "cleanup" service is responding slowly.  Perhaps your filesystem
is slow, since cleanup has to commit the message to disk.  Or some
tables you're using in cleanup are slow.  The verbose logging does
not help, it puts pressure on the filesystem...

-- 
Viktor.

Re: Limiting global number of outgoing connections

[Viktor Dukhovni]

> On Jan 4, 2019, at 7:23 AM, Wietse Venema  wrote:
> 
> For one delivery request, the Postfix SMTP client gets up to
> five ($smtp_mx_address_limit) addresses from DNS, and burns through
> those addresses in a split second as they fail with ECONNREFUSED.
> Instantly, he has used up his entire 5 connections/second budget
> with only one delivert request.
> 
> Obviously, the queue scheduler has no idea how many 'connections'
> are being used up per delivery request. No amount of scheduler
> tweaking will change that.

Well for that we have smtp_mx_address_limit, which could be
reduced from 5 to 2 (or perhaps 3).  That does risk problems
delivering to sites whose primary MX has multiple IPs and is
fake, with the real MX at worse priority, but such sites clearly
don't really want to receive much mail... :-(

-- 
Viktor.

Re: Slowness after upgrading from postfix 2.x to 3.1.8

[Christopher R. Gabriel]

On Fri, 2019-01-04 at 09:49 -0500, Viktor Dukhovni wrote:
> > On Jan 4, 2019, at 9:23 AM, Christopher R. Gabriel <
> > christopher.gabr...@gmail.com> wrote:
> > 
> > Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]: rec_put: type
> > E
> > len 0 data 
> > Nov 30 09:11:31 postfix01 postfix-main/smtpd[31800]:
> > vstream_fflush_some: fd 18 flush 2433
> > Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]:
> > vstream_buf_get_ready: fd 18 got 18
> > Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: public/cleanup
> > socket: wanted attribute: status
> 
> Your "cleanup" service is responding slowly.  Perhaps your filesystem
> is slow, since cleanup has to commit the message to disk. 

All spool/data dirs are on ramdisk tmpfs, that's why I'm wondering.

>  Or some tables you're using in cleanup are slow. 

I only have a header_checks table with 1 single rule to log a specific
header, and a transport map redis-based. Exactly the same configuration
I have on postfix 2.x.

>  The verbose logging does not help, it puts pressure on the
> filesystem...

I've enable the verbose logging only for debug pourposes, I find the
same behaviour even with logging disabled.

Re: bypass policy server in recipient_restrictions when subject contains string

[Bill Cole]

On 4 Jan 2019, at 9:36, Stefan Bauer wrote:

is there a way to bypass policy server in smtp_recipient_restrictions, 
in

case, subject contains special string?


No. As documented, smtp_recipient_restrictions is evaluated for each 
RCPT command, all of which occur before the DATA command, which is when 
the SMTP server can examine message data (including message headers.)

Re: Slowness after upgrading from postfix 2.x to 3.1.8

[Viktor Dukhovni]

> On Jan 4, 2019, at 10:04 AM, Christopher R. Gabriel 
>  wrote:
> 
>> Or some tables you're using in cleanup are slow. 
> 
> I only have a header_checks table with 1 single rule to log a specific
> header, and a transport map redis-based. Exactly the same configuration
> I have on postfix 2.x.

Cleanup does perform transport lookups.  Is redis performing well?

>> The verbose logging does not help, it puts pressure on the
>> filesystem...
> 
> I've enable the verbose logging only for debug pourposes, I find the
> same behaviour even with logging disabled.

Well, since cleanup is slow, move the verbose logging to cleanup if
you can't determine why cleanup is blocked more directly.

Cleanup performs transport lookups, canonical and virtual rewriting,
header/body checks, writes and syncs the message to disk, and
sends a 1-byte notice to the queue manager socket.  With inflow_delay
it also waits that long to receive a token from the queue manager.

One of these things is slow.

-- 
Viktor.

Re: bypass policy server in recipient_restrictions when subject contains string

[Stefan Bauer]

Would it be possible to have FILTER as action in policy server (in
recipient_restrictions) and send it to smtp process that uses header_checks
do have mailroute based on subject?




Am Fr., 4. Jan. 2019 um 16:08 Uhr schrieb Bill Cole <
postfixlists-070...@billmail.scconsult.com>:

> On 4 Jan 2019, at 9:36, Stefan Bauer wrote:
>
> > is there a way to bypass policy server in smtp_recipient_restrictions,
> > in
> > case, subject contains special string?
>
> No. As documented, smtp_recipient_restrictions is evaluated for each
> RCPT command, all of which occur before the DATA command, which is when
> the SMTP server can examine message data (including message headers.)
>

Re: policy server, TLS only exeptions and restrictions

[Viktor Dukhovni]

> On Jan 4, 2019, at 9:10 AM, Matus UHLAR - fantomas  wrote:
> 
> this looks to me that you search for connection between 
> smtpd_recipient_restrictions
> and smtp_tls_policy_maps, and there is none.
> 
> the "check_policy_service private/policy" communicates via unix socket
> private/policy (apparetly in postfix directory) to external program that
> tells smtpd what to do.
> 
> if you want your policy server to return dunno for sending domain
> "remote-site.de", your policy server must look to the /etc/postfix/tls/finance
> table for the remote-site.de domain.
> 
> the policy server doesn't look to your "smtp_tls_policy_maps" settings,
> usually it does not read postfix configuration at all.

This is where recipient verification has an advantage over a policy
service.  For SASL authenticated users, who can relay outbound, the
OP could replace the policy service with a recipient verification
callout:

   smtpd_relay_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination

   smtpd_recipient_restrictions
permit_auth_destination,
reject_unverified_recipient

This *is* sensitive to outbound TLS policy, because recipient
verification uses outbound SMTP connections to probe for TLS
support, and will fail where TLS is mandated and not available.

Of course static configuration that are reflected in both the
policy service and the SMTP TLS policy yield more predictable,
if not always up to date behaviour.

-- 
Viktor.

Re: Slowness after upgrading from postfix 2.x to 3.1.8

[Christopher R. Gabriel]

On Fri, 2019-01-04 at 10:26 -0500, Viktor Dukhovni wrote:
> > On Jan 4, 2019, at 10:04 AM, Christopher R. Gabriel <
> > christopher.gabr...@gmail.com> wrote:
> > 
> > > Or some tables you're using in cleanup are slow. 
> > 
> > I only have a header_checks table with 1 single rule to log a
> > specific
> > header, and a transport map redis-based. Exactly the same
> > configuration
> > I have on postfix 2.x.
> 
> Cleanup does perform transport lookups.  Is redis performing well?

Yes, blazing fast. It's shown in the full log of the smtp session, al
the whole transaction (until the reported slowness event) is immediate.

> > > The verbose logging does not help, it puts pressure on the
> > > filesystem...
> > 
> > I've enable the verbose logging only for debug pourposes, I find
> > the
> > same behaviour even with logging disabled.
> 
> Well, since cleanup is slow, move the verbose logging to cleanup if
> you can't determine why cleanup is blocked more directly.

I'll try this too.

> Cleanup performs transport lookups, canonical and virtual rewriting,
> header/body checks, writes and syncs the message to disk, and
> sends a 1-byte notice to the queue manager socket.  With inflow_delay
> it also waits that long to receive a token from the queue manager.

in_flow_delay is default, like in the other postfix 2.x instances, all
data (both new and old server) is on ramdisk.. I'll dig into the
verbose log for cleanup, hoping to find the problem

Re: bypass policy server in recipient_restrictions when subject contains string

[Bill Cole]

On 4 Jan 2019, at 10:36, Stefan Bauer wrote:


Would it be possible to have FILTER as action in policy server


Yes, but FILTER behaves as documented in the access(5) man page. The 
first 5 words there describing what FILTER does are critical, but you 
should read it all...



(in
recipient_restrictions) and send it to smtp process that uses 
header_checks

do have mailroute based on subject?


There can be NO WAY to exempt a message from policy that would apply at 
RCPT time with facts that cannot be known until end-of-DATA time. 
Postfix cannot modify the basic constraints of non-quantum causality or 
the arrow of time or tell SMTP clients to re-order the fixed command 
sequence of SMTP.


If you want to make any decisions about a message based on a header, you 
must do that with a tool (header_checks, milter, content_filter, or 
post-delivery backend) that has access to the message data because it 
operates at end-of-DATA or after queueing.

Content filter - reijnect message back into queue

[Rafael Azevedo]

Hi there,

I'm trying to build my own content filter so I can actually filter outgoing
messages and take appropriated actions upon spam messages.

After some time I was able to make postfix send messages to the content
filter.

The documentation says that content_filter expects a "transport:maps"
response.

The content_filter configuration parameter expects a value of the form
> transport:destination. The transport name specifies the first field of a
> mail delivery agent definition in master.cf; the syntax of the next-hop
> destination is described in the manual page of the corresponding delivery
> agent.


My script just returns "ACCEPT" as the content_filter action.

The thing is that the messages are then gone after the content_filter
receives it.

Looking my log files:

Jan  4 13:58:19 lab postfix/pipe[2025749]: 193EA13DB044: to=,
> orig_to=, relay=post_queue_content_filter, delay=0.11,
> delays=0.09/0/0/0.02, dsn=2.0.0, status=sent (delivered via
> post_queue_content_filter service (action=PERMIT))


After that, the message is gone.

So I kindly ask you guys how can I re-inject message back into queue.

Another question is how to postpone a specific message when re-injecting it
to queue. For example: the filter will accept the message but hold it for 5
minutes before delivering it to final destination.

[master.cf]

smtp  inet  n   -   n   -   30  smtpd
>   -o content_filter=post_queue_content_filter:dummy



post_queue_content_filterunix-   n   n   -   -
>  pipe

user=iagente argv=/home/postfix/app/tools/contentFilter.php
> /usr/sbin/sendmail -oi -f ${sender} ${recipient}


Note that I've added at the last line a sendmail call. That was copied from
another server running spamassassin (which ijnects message back to queue
after content analysis). I'm still trying to understand the right way to do
it.

Any help would be much appreciated.

Thanks in advance.

BR,

Rafael

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

> On Jan 4, 2019, at 11:54 AM, Rafael Azevedo  wrote:
> 
> So I kindly ask you guys how can I re-inject message back into queue.

http://www.postfix.org/FILTER_README.html

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Hi Viktor,

Thanks for the tip.

I did my best within my knowledges to archive that goal.

After reading that document so many times this was the farther I could go.

Another tip would be much appreciated.

BR,

Em sex, 4 de jan de 2019 às 15:28, Viktor Dukhovni <
postfix-us...@dukhovni.org> escreveu:

>
>
> > On Jan 4, 2019, at 11:54 AM, Rafael Azevedo  wrote:
> >
> > So I kindly ask you guys how can I re-inject message back into queue.
>
> http://www.postfix.org/FILTER_README.html
>
> --
> Viktor.
>
>

[Partially solved] Re: Address rewriting not working

[Celejar]

On Wed, 2 Jan 2019 19:58:18 -0500
Viktor Dukhovni  wrote:

> > On Jan 2, 2019, at 7:12 PM, Celejar  wrote:
> > 
> > I'm configuring Postfix to relay mail via a smarthost, and I need to
> > rewrite the sender address in order for the smarthost to accept the
> > mail (and not reject it as 'relaying'). I'm using generic mapping to do
> > this, and it works correctly on two of my systems (Debian Sid,
> > running Postfix 3.3.2), but not on a third (Debian Stretch, running
> > 3.1.8). I've tried all sorts of adjustments and debugging, and I'm at
> > my wits' end. Below is the configuration and logging from the broken
> > system:
> 

> Because the queue manager logs the envelope sender prior to delivery,
> it always logs the original value, and recipient logging in delivery
> agents is also the form before generic rewriting.  The output of
> generic rewriting is not logged on the sending system (except perhaps
> in verbose logging that should not normally be enabled).

Okay, by testing with swaks I've confirmed the suspicion that I broached
in my previous mail: on the problematic system, the rewrite of the email
header 'From: root' is to 'From:  (root)', which
causes the mail to be rejected by Zoho's server with '553 Relaying
disallowed as @'. On the working systems, the rewrite is to a more
normal 'From: root ', which Zoho accepts.

So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
Postfix?

Celejar

Re: [Partially solved] Re: Address rewriting not working

[Wietse Venema]

Celejar:
> Okay, by testing with swaks I've confirmed the suspicion that I broached
> in my previous mail: on the problematic system, the rewrite of the email
> header 'From: root' is to 'From:  (root)', which
> causes the mail to be rejected by Zoho's server with '553 Relaying
> disallowed as @'. On the working systems, the rewrite is to a more
> normal 'From: root ', which Zoho accepts.
> 
> So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> Postfix?

See: http://www.postfix.org/postconf.5.htnl#header_from_format

Wietse

header_from_format (default: standard)
   The  format of the Postfix-generated From: header. This setting affects
   the appearance of 'full name' information when a local program such  as
   /bin/mail  submits  a  message without From: header through the Postfix
   sendmail(1) command.

   Specify one of the following:

   standard (default)
  Produce a header formatted as "From: name ".   This  is
  the default as of Postfix 3.3.

   obsolete
  Produce  a  header  formatted as "From: address (name)". This is
  the behavior prior to Postfix 3.3.

   Notes:

   o  Postfix generates the format "From: address" when name  informa-
  tion  is  unavailable  or  the envelope sender address is empty.
  This is the same behavior as prior to Postfix 3.3.

   o  In the standard form, the name will be  quoted  if  it  contains
  specials  as defined in RFC 5322, or the "!%" address operators.

   o  The Postfix sendmail(1) command gets name information  from  the
  -F  command-line  option, from the NAME environment variable, or
  from the UNIX password file.

   This feature is available in Postfix 3.3 and later.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Viktor,

After doing as explained @ http://www.postfix.org/FILTER_README.html, I'm
still having same behavior.

Jan  4 15:23:16 zimuslab postfix/pipe[2026324]: 90FAD13DE3B1: to=,
relay=post_queue_content_filter, delay=0.07, delays=0.04/0/0/0.02,
dsn=2.0.0, status=sent (delivered via post_queue_content_filter service
(action=PERMIT))

After changing my script to simply "return" the situation persists.

Jan  4 15:33:27 zimuslab postfix/pipe[2026361]: 7751713DE600: to=,
relay=post_queue_content_filter, delay=0.07, delays=0.04/0/0/0.02,
dsn=2.0.0, status=sent (delivered via post_queue_content_filter service)

BR,
Rafael

Em sex, 4 de jan de 2019 às 15:28, Viktor Dukhovni <
postfix-us...@dukhovni.org> escreveu:

>
>
> > On Jan 4, 2019, at 11:54 AM, Rafael Azevedo  wrote:
> >
> > So I kindly ask you guys how can I re-inject message back into queue.
>
> http://www.postfix.org/FILTER_README.html
>
> --
> Viktor.
>
>

RE: Content filter - reijnect message back into queue

[Luis Miguel Flores dos Santos]

Try create another postfix instance and force your filter send message to it.



De: owner-postfix-us...@postfix.org  em nome 
de Rafael Azevedo 
Enviado: sexta-feira, 4 de janeiro de 2019 16:21
Para: Postfix users
Assunto: Re: Content filter - reijnect message back into queue

Viktor,

After doing as explained @ 
http://www.postfix.org/FILTER_README.html,
 I'm still having same behavior.

Jan  4 15:23:16 zimuslab postfix/pipe[2026324]: 90FAD13DE3B1: 
to=mailto:x...@xx.net>>, relay=post_queue_content_filter, 
delay=0.07, delays=0.04/0/0/0.02, dsn=2.0.0, status=sent (delivered via 
post_queue_content_filter service (action=PERMIT))

After changing my script to simply "return" the situation persists.

Jan  4 15:33:27 zimuslab postfix/pipe[2026361]: 7751713DE600: 
to=mailto:x...@xx.net>>, relay=post_queue_content_filter, 
delay=0.07, delays=0.04/0/0/0.02, dsn=2.0.0, status=sent (delivered via 
post_queue_content_filter service)

BR,
Rafael

Em sex, 4 de jan de 2019 às 15:28, Viktor Dukhovni 
mailto:postfix-us...@dukhovni.org>> escreveu:


> On Jan 4, 2019, at 11:54 AM, Rafael Azevedo 
> mailto:raf...@gmail.com>> wrote:
>
> So I kindly ask you guys how can I re-inject message back into queue.

http://www.postfix.org/FILTER_README.html

--
Viktor.

Re: Content filter - reijnect message back into queue

[Wietse Venema]

Rafael Azevedo:
> Looking my log files:
> 
> Jan  4 13:58:19 lab postfix/pipe[2025749]: 193EA13DB044: to=,
> > orig_to=, relay=post_queue_content_filter, delay=0.11,
> > delays=0.09/0/0/0.02, dsn=2.0.0, status=sent (delivered via
> > post_queue_content_filter service (action=PERMIT))
> 
> After that, the message is gone.

You forgot to send it back into Postfix.

BTW what is that "action=PERMIT" stuff? There is no such feature
with Postfix filters. Are you confusing SMTPD_POLICY_README and
FILTER_README?

> So I kindly ask you guys how can I re-inject message back into queue.

FILTER_README gives multiple examples for doing exactly that. One
approach uses /usr/sbin/sendmail, and one uses SMTP.

SMTPD_POLICY_README is about sending name=value pairs and receiving
a response with action=(PERMIT or some other action).

Wietse

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Hi Wietse,

Thanks for your help.

> You forgot to send it back into Postfix.

Would you please tell me how to send it back to POSTFIX ?

> BTW what is that "action=PERMIT" stuff? There is no such feature
> with Postfix filters. Are you confusing SMTPD_POLICY_README and
> FILTER_README?

Yes, I tried to use same syntax of "check_policy_service" but had no success.

Thanks a lot.

BR,

Rafael

Re: Slowness after upgrading from postfix 2.x to 3.1.8

[Matus UHLAR - fantomas]

On 04.01.19 15:23, Christopher R. Gabriel wrote:

I have a generator server which injects (via smtp) into postfix, the
actual sender, and when burst of delivery happens, the receiving
postfix stuck before answering to the generator, which causes the
generator queues to fill up.



Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: abort all milters
Nov 30 09:11:58 postfix01 postfix-main/smtpd[31800]: milter8_abort:
abort milter inet:127.0.0.1:12301




postfix01 data/spool are on tmpfs.


are you OK with losing mail when something breaks?


smtpd_milters = inet:127.0.0.1:12301


is this milter running properly?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam is for losers who can't get business any other way.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Digging on FILTER_README, I've noticed that:

> The content_filter setting has lower precedence than a FILTER
action that is specified in an access(5), header_checks(5) or
body_checks(5) table.

Even using OK, PREPEND, the message could not go back into queue.

JIT:

ACCEPT ACTIONS
   OK Accept the address etc. that matches the pattern.

   all-numerical
  An  all-numerical result is treated as OK. This format is gener-
  ated  by  address-based  relay  authorization  schemes  such  as
  pop-before-smtp.

   For other accept actions, see "OTHER ACTIONS" below.

REJECT ACTIONS
   Postfix  version 2.3 and later support enhanced status codes as defined
   in RFC 3463.  When no code is specified at the beginning  of  the  text
   below, Postfix inserts a default enhanced status code of "5.7.1" in the
   case of reject actions, and "4.7.1" in the case of defer  actions.  See
   "ENHANCED STATUS CODES" below.

   4NN text

   5NN text
  Reject  the  address  etc. that matches the pattern, and respond
  with the numerical three-digit code and  text.  4NN  means  "try
  again later", while 5NN means "do not try again".

  The  following  responses  have  special meaning for the Postfix
  SMTP server:

  421 text (Postfix 2.3 and later)

  521 text (Postfix 2.6 and later)
 After responding with the numerical three-digit code  and
 text,  disconnect immediately from the SMTP client.  This
 frees up SMTP server resources so that they can  be  made
 available to another SMTP client.

 Note: The "521" response should be used only with botnets
 and other malware where interoperability is  of  no  con-
 cern.   The  "send  521  and  disconnect" behavior is NOT
 defined in the SMTP standard.

   REJECT optional text...
  Reject the address etc. that matches  the  pattern.  Reply  with
  "$access_map_reject_code  optional  text..."  when  the optional
  text is specified, otherwise reply with a generic error response
  message.

   DEFER optional text...
  Reject  the  address  etc.  that matches the pattern. Reply with
  "$access_map_defer_code optional text..." when the optional text
  is specified, otherwise reply with a generic error response mes-
  sage.

  This feature is available in Postfix 2.6 and later.

   DEFER_IF_REJECT optional text...
  Defer the request if some later restriction would  result  in  a
  REJECT action. Reply with "$access_map_defer_code 4.7.1 optional
  text..." when the optional text is  specified,  otherwise  reply
  with a generic error response message.

  Prior to Postfix 2.6, the SMTP reply code is 450.

  This feature is available in Postfix 2.1 and later.

   DEFER_IF_PERMIT optional text...
  Defer the request if some later restriction would result in a an
  explicit   orimplicitPERMITaction. Replywith
  "$access_map_defer_code   4.7.1optional  text..."  when  the
  optional text is specified, otherwise reply with a generic error
  response message.

  Prior to Postfix 2.6, the SMTP reply code is 450.

  This feature is available in Postfix 2.1 and later.

   For other reject actions, see "OTHER ACTIONS" below.

OTHER ACTIONS
   restriction...
  Applythe   named   UCE   restriction(s)   (permit,   reject,
  reject_unauth_destination, and so on).

   BCC user@domain
  Send one copy of the message to the specified recipient.

  If multiple BCC actions are specified within the same SMTP  MAIL
  transaction, with Postfix 3.0 only the last action will be used.

  This feature is available in Postfix 3.0 and later.

   DISCARD optional text...
  Claim successful delivery and silently discard the message.  Log
  the optional text if specified, otherwise log a generic message.

  Note: this action currently affects all recipients of  the  mes-
  sage.   To  discard  only  one  recipient without discarding the
  entire message, use the transport(5) table to direct mail to the
  discard(8) service.

  This feature is available in Postfix 2.0 and later.

   DUNNO  Pretend that the lookup key was not found. This prevents Postfix
  from trying substrings of the lookup key (such  as  a  subdomain
  name, or a network address subnetwork).

  This feature is available in P

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Another attempt:

Jan  4 16:39:21 lab postfix/pipe[2026654]: 82B8813DF90D:
to=, relay=post_queue_content_filter, delay=0.07,
delays=0.04/0/0/0.03, dsn=2.0.0, status=sent (delivered via
post_queue_content_filter service (action=FILTER localhost:10026))

NOTE:
action=FILTER localhost:10026

The content_filter setting has lower precedence than a FILTER
action that is specified in an access(5), header_checks(5) or
body_checks(5) table.

As described in ACCESS (5):

FILTER transport:destination
  After the message is queued, send the entire message through the
  specified external content filter. The transport name  specifies
  the  first  field  of  a  mail delivery agent definition in mas-
  ter.cf; the syntax of the next-hop destination is  described  in
  the  manual  page  of  the  corresponding  delivery agent.  More
  information about external content filters  is  in  the  Postfix
  FILTER_README file.

  Note  1: do not use $number regular expression substitutions for
  transport or destination unless you know  that  the  information
  has a trusted origin.

  Note  2:  this  action overrides the main.cf content_filter set-
  ting, and affects all recipients of the  message.  In  the  case
  that  multiple  FILTER  actions  fire, only the last one is exe-
  cuted.

  Note 3: the purpose of the FILTER command is to override message
  routing.   To  override  the  recipient's  transport but not the
  next-hop destination, specify an empty filter destination (Post-
  fix  2.7  and  later),  or  specify a transport:destination that
  delivers through a different Postfix instance (Postfix  2.6  and
  earlier). Other options are using the recipient-dependent trans-
  port_maps  or  the  sender-dependent   sender_dependent_default-
  _transport_maps features.

  This feature is available in Postfix 2.0 and later.

What am I missing here?

Thanks!

BR,

Rafael

Em sex, 4 de jan de 2019 às 16:57, Rafael Azevedo  escreveu:
>
> Digging on FILTER_README, I've noticed that:
>
> > The content_filter setting has lower precedence than a FILTER
> action that is specified in an access(5), header_checks(5) or
> body_checks(5) table.
>
> Even using OK, PREPEND, the message could not go back into queue.
>
> JIT:
>
> ACCEPT ACTIONS
>OK Accept the address etc. that matches the pattern.
>
>all-numerical
>   An  all-numerical result is treated as OK. This format is gener-
>   ated  by  address-based  relay  authorization  schemes  such  as
>   pop-before-smtp.
>
>For other accept actions, see "OTHER ACTIONS" below.
>
> REJECT ACTIONS
>Postfix  version 2.3 and later support enhanced status codes as defined
>in RFC 3463.  When no code is specified at the beginning  of  the  text
>below, Postfix inserts a default enhanced status code of "5.7.1" in the
>case of reject actions, and "4.7.1" in the case of defer  actions.  See
>"ENHANCED STATUS CODES" below.
>
>4NN text
>
>5NN text
>   Reject  the  address  etc. that matches the pattern, and respond
>   with the numerical three-digit code and  text.  4NN  means  "try
>   again later", while 5NN means "do not try again".
>
>   The  following  responses  have  special meaning for the Postfix
>   SMTP server:
>
>   421 text (Postfix 2.3 and later)
>
>   521 text (Postfix 2.6 and later)
>  After responding with the numerical three-digit code  and
>  text,  disconnect immediately from the SMTP client.  This
>  frees up SMTP server resources so that they can  be  made
>  available to another SMTP client.
>
>  Note: The "521" response should be used only with botnets
>  and other malware where interoperability is  of  no  con-
>  cern.   The  "send  521  and  disconnect" behavior is NOT
>  defined in the SMTP standard.
>
>REJECT optional text...
>   Reject the address etc. that matches  the  pattern.  Reply  with
>   "$access_map_reject_code  optional  text..."  when  the optional
>   text is specified, otherwise reply with a generic error response
>   message.
>
>DEFER optional text...
>   Reject  the  address  etc.  that matches the pattern. Reply with
>   "$access_map_defer_code optional text..." when the optional text
>   is specified, otherwise reply with a generic error response mes-
>   sage.
>
>   This feature is available in Postfix

Re: Content filter - reijnect message back into queue

[Wietse Venema]

Rafael Azevedo:
> Hi Wietse,
> 
> Thanks for your help.
> 
> > You forgot to send it back into Postfix.
> 
> Would you please tell me how to send it back to POSTFIX ?

FILTER_README has examples for doing that with /usr/sbin/sendmail
and with SMTP.

Wietse

Re: Content filter - reijnect message back into queue

[Wietse Venema]

Rafael Azevedo:
> Another attempt:
> 
> Jan  4 16:39:21 lab postfix/pipe[2026654]: 82B8813DF90D:
> to=, relay=post_queue_content_filter, delay=0.07,
> delays=0.04/0/0/0.03, dsn=2.0.0, status=sent (delivered via
> post_queue_content_filter service (action=FILTER localhost:10026))

THAT IS POLICY DELEGATION PROTOCOL NOT CONTENT FILTER.

Re: [Partially solved] Re: Address rewriting not working

[Celejar]

On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
Wietse Venema  wrote:

> Celejar:
> > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > in my previous mail: on the problematic system, the rewrite of the email
> > header 'From: root' is to 'From:  (root)', which
> > causes the mail to be rejected by Zoho's server with '553 Relaying
> > disallowed as @'. On the working systems, the rewrite is to a more
> > normal 'From: root ', which Zoho accepts.
> > 
> > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > Postfix?
> 
> See: http://www.postfix.org/postconf.5.htnl#header_from_format

...

> header_from_format (default: standard)

...

>Specify one of the following:
> 
>standard (default)
>   Produce a header formatted as "From: name ".   This  is
>   the default as of Postfix 3.3.
> 
>obsolete
>   Produce  a  header  formatted as "From: address (name)". This is
>   the behavior prior to Postfix 3.3.

...

>This feature is available in Postfix 3.3 and later.

And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
provider, and this feature isn't available ;) So I guess I'm stuck,
unless I can upgrade Postfix?

Thanks for the pointer,
Celejar

Re: [Partially solved] Re: Address rewriting not working

[Viktor Dukhovni]

> On Jan 4, 2019, at 2:56 PM, Celejar  wrote:
> 
> And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> provider, and this feature isn't available ;) So I guess I'm stuck,
> unless I can upgrade Postfix?

Your other option, if possible, is to inject email into Postfix
with the "From:" header already constructed the way you want.
Postfix only adds "From:" headers when missing.

You could also use a PCRE REPLACE rule in smtp_header_checks:

# Replace some legacy "address (display name)" forms with a
# more modern "display name " form.
/^From:\s*([^\s<>]+)\s+\(([^"]*)\)\s*$/ REPLACE From: "$2" <$1>

The server that's accepting "From: display name "
and rejecting "From: address (display name)" is not blameless
It should be able to process either form.

-- 
-- 
Viktor.

Re: [Partially solved] Re: Address rewriting not working

[Wietse Venema]

Celejar:
> On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
> Wietse Venema  wrote:
> 
> > Celejar:
> > > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > > in my previous mail: on the problematic system, the rewrite of the email
> > > header 'From: root' is to 'From:  (root)', which
> > > causes the mail to be rejected by Zoho's server with '553 Relaying
> > > disallowed as @'. On the working systems, the rewrite is to a more
> > > normal 'From: root ', which Zoho accepts.
> > > 
> > > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > > Postfix?
> > 
> > See: http://www.postfix.org/postconf.5.htnl#header_from_format
> >This feature is available in Postfix 3.3 and later.
> 
> And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> provider, and this feature isn't available ;) So I guess I'm stuck,
> unless I can upgrade Postfix?

Can you provide the From: header *before* mail enters Postfix?

Otherwise you will have to update to Postfix 3.3.

Wietse

Re: Content filter - reijnect message back into queue

[Matus UHLAR - fantomas]

You forgot to send it back into Postfix.


On 04.01.19 16:47, Rafael Azevedo wrote:

Would you please tell me how to send it back to POSTFIX ?


call sendmail and pass te message to it, or sent it to postfix via
SMTP/LMTP, apparently on different port where content_filter is turned off,
so postfix doesn't send it to your content filter again.

it's explained in http://www.postfix.org/FILTER_README.html#principles
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

> FILTER_README has examples for doing that with /usr/sbin/sendmail
> and with SMTP.


Has anybody ever made that example work?

There's no way.

post_queue_content_filterunix-   n   n   -   -
  pipe
flags=Rq user=myuser null_sender=
argv=/home/postfix/app/tools/postfix-filter.sh -f ${sender} --
${recipient}


/home/postfix/app/tools/postfix-filter.sh:
-
#!/bin/sh
# Simple shell-based filter. It is meant to be invoked as follows:
#   /path/to/script -f sender recipients...

# Localize these. The -G option does nothing before Postfix 2.3.
INSPECT_DIR=/var/spool/filter
SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.

# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69

# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15

# Start processing.
cd $INSPECT_DIR || {
echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }

cat >in.$$ || {
echo Cannot save mail to file; exit $EX_TEMPFAIL; }

# Specify your content filter here.
# filter , relay=post_queue_content_filter, delay=1.1,
delays=0.04/0/0/1, dsn=4.3.0, status=deferred (temporary failure.
Command output: postdrop: error: untrusted configuration directory
name: /etc/postfix/ postdrop: fatal: specify
"alternate_config_directories = /etc/postfix/" in /etc/postfix/main.cf
sendmail: warning: command "/usr/sbin/postdrop -r" exited with status
1 sendmail: fatal: x...@xx.net(5001): unable to execute
/usr/sbin/postdrop -r: Success )



Thanks!

BR,
Rafael

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Hi Matus,

Thanks a lot for the help.

I tried setting the FILTER to localhost:otherport-with-no-filter but
had same behavior.

action=FILTER localhost:10026

no success.

Em sex, 4 de jan de 2019 às 18:36, Matus UHLAR - fantomas
 escreveu:
>
> >> You forgot to send it back into Postfix.
>
> On 04.01.19 16:47, Rafael Azevedo wrote:
> >Would you please tell me how to send it back to POSTFIX ?
>
> call sendmail and pass te message to it, or sent it to postfix via
> SMTP/LMTP, apparently on different port where content_filter is turned off,
> so postfix doesn't send it to your content filter again.
>
> it's explained in http://www.postfix.org/FILTER_README.html#principles
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Emacs is a complicated operating system without good text editor.

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

> On Jan 4, 2019, at 3:37 PM, Rafael Azevedo  wrote:
> 
> Jan  4 17:41:54 lab postfix/pipe[2027085]: EE5D013E179F:
> to=, relay=post_queue_content_filter, delay=1.1,
> delays=0.04/0/0/1, dsn=4.3.0, status=deferred (temporary failure.
> Command output: postdrop: error: untrusted configuration directory
> name: /etc/postfix/ postdrop: fatal: specify
> "alternate_config_directories = /etc/postfix/" in /etc/postfix/main.cf
> sendmail: warning: command "/usr/sbin/postdrop -r" exited with status
> 1 sendmail: fatal: x...@xx.net(5001): unable to execute
> /usr/sbin/postdrop -r: Success )

DO NOT set "config_directory = /etc/postfix/" in main.cf.  In fact do
do not set it at all.

-- 
Viktor.

Re: [Partially solved] Re: Address rewriting not working

[Celejar]

On Fri, 4 Jan 2019 15:22:08 -0500 (EST)
Wietse Venema  wrote:

> Celejar:
> > On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
> > Wietse Venema  wrote:
> > 
> > > Celejar:
> > > > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > > > in my previous mail: on the problematic system, the rewrite of the email
> > > > header 'From: root' is to 'From:  (root)', which
> > > > causes the mail to be rejected by Zoho's server with '553 Relaying
> > > > disallowed as @'. On the working systems, the rewrite is to a more
> > > > normal 'From: root ', which Zoho accepts.
> > > > 
> > > > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > > > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > > > Postfix?
> > > 
> > > See: http://www.postfix.org/postconf.5.htnl#header_from_format
> > >This feature is available in Postfix 3.3 and later.
> > 
> > And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> > provider, and this feature isn't available ;) So I guess I'm stuck,
> > unless I can upgrade Postfix?
> 
> Can you provide the From: header *before* mail enters Postfix?
> 
> Otherwise you will have to update to Postfix 3.3.

I suppose I might be able to look at individual mail sending
applications to see what they can do (cron, etc.), but it will probably
be simpler and more robust to just try to use a newer Postfix. I can't
easily upgrade the version on the current system, since it's running
Debian Stable, and the version from Unstable has unavailable
dependencies, so I'll probably just run a more recent Postfix on a
virtual machine.

Thanks for the help,

Celejar

Re: Content filter - reijnect message back into queue

[Matus UHLAR - fantomas]

On 04.01.19 18:41, Rafael Azevedo wrote:

Thanks a lot for the help.

I tried setting the FILTER to localhost:otherport-with-no-filter but
had same behavior.

action=FILTER localhost:10026


where did you set it? it's your own filter, it's not postfix, I have no idea
what you need to set ...


Em sex, 4 de jan de 2019 às 18:36, Matus UHLAR - fantomas
 escreveu:


>> You forgot to send it back into Postfix.

On 04.01.19 16:47, Rafael Azevedo wrote:
>Would you please tell me how to send it back to POSTFIX ?

call sendmail and pass te message to it, or sent it to postfix via
SMTP/LMTP, apparently on different port where content_filter is turned off,
so postfix doesn't send it to your content filter again.

it's explained in http://www.postfix.org/FILTER_README.html#principles

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Fucking windows! Bring Bill Gates! (Southpark the movie)

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

In fact I didn't.
There's no config_directory set in my main.cf file.

Em sex, 4 de jan de 2019 às 18:46, Viktor Dukhovni
 escreveu:
>
> > On Jan 4, 2019, at 3:37 PM, Rafael Azevedo  wrote:
> >
> > Jan  4 17:41:54 lab postfix/pipe[2027085]: EE5D013E179F:
> > to=, relay=post_queue_content_filter, delay=1.1,
> > delays=0.04/0/0/1, dsn=4.3.0, status=deferred (temporary failure.
> > Command output: postdrop: error: untrusted configuration directory
> > name: /etc/postfix/ postdrop: fatal: specify
> > "alternate_config_directories = /etc/postfix/" in /etc/postfix/main.cf
> > sendmail: warning: command "/usr/sbin/postdrop -r" exited with status
> > 1 sendmail: fatal: x...@xx.net(5001): unable to execute
> > /usr/sbin/postdrop -r: Success )
>
> DO NOT set "config_directory = /etc/postfix/" in main.cf.  In fact do
> do not set it at all.
>
> --
> Viktor.
>

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

> On Jan 4, 2019, at 4:59 PM, Rafael Azevedo  wrote:
> 
>>> Jan  4 17:41:54 lab postfix/pipe[2027085]: EE5D013E179F:
>>> to=, relay=post_queue_content_filter, delay=1.1,
>>> delays=0.04/0/0/1, dsn=4.3.0, status=deferred (temporary failure.
>>> Command output: postdrop: error: untrusted configuration directory
>>> name: /etc/postfix/ postdrop: fatal: specify
>>> "alternate_config_directories = /etc/postfix/" in /etc/postfix/main.cf
>>> sendmail: warning: command "/usr/sbin/postdrop -r" exited with status
>>> 1 sendmail: fatal: x...@xx.net(5001): unable to execute
>>> /usr/sbin/postdrop -r: Success )
>> 
>> DO NOT set "config_directory = /etc/postfix/" in main.cf.  In fact do
>> do not set it at all.
> 
> In fact I didn't.
> There's no config_directory set in my main.cf file.

The logs don't lie.  Either main.cf has "config_directory = /etc/postfix/"
with a trailing "/", or Postfix was started via "postfix -c /etc/postfix/ start"
or you have enabled multiple instances, and the multi-instance configuration
lists "/etc/postfix/", ... or similar.

By the time "postdrop" is running in your script the "MAIL_CONFIG" environment
variable is set to "/etc/postfix/" rather than just "/etc/postfix".

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

> THAT IS POLICY DELEGATION PROTOCOL NOT CONTENT FILTER.

What are the differences?

Re: policy server, TLS only exeptions and restrictions

[Stefan Bauer]

great idea, but recipient verification is not something, remote servers
like.really like.

Am Freitag, 4. Januar 2019 schrieb Viktor Dukhovni <
postfix-us...@dukhovni.org>:
>> On Jan 4, 2019, at 9:10 AM, Matus UHLAR - fantomas 
wrote:
>>
>> this looks to me that you search for connection between
smtpd_recipient_restrictions
>> and smtp_tls_policy_maps, and there is none.
>>
>> the "check_policy_service private/policy" communicates via unix socket
>> private/policy (apparetly in postfix directory) to external program that
>> tells smtpd what to do.
>>
>> if you want your policy server to return dunno for sending domain
>> "remote-site.de", your policy server must look to the
/etc/postfix/tls/finance
>> table for the remote-site.de domain.
>>
>> the policy server doesn't look to your "smtp_tls_policy_maps" settings,
>> usually it does not read postfix configuration at all.
>
> This is where recipient verification has an advantage over a policy
> service.  For SASL authenticated users, who can relay outbound, the
> OP could replace the policy service with a recipient verification
> callout:
>
>smtpd_relay_restrictions =
> permit_mynetworks,
> permit_sasl_authenticated,
> reject_unauth_destination
>
>smtpd_recipient_restrictions
> permit_auth_destination,
> reject_unverified_recipient
>
> This *is* sensitive to outbound TLS policy, because recipient
> verification uses outbound SMTP connections to probe for TLS
> support, and will fail where TLS is mandated and not available.
>
> Of course static configuration that are reflected in both the
> policy service and the SMTP TLS policy yield more predictable,
> if not always up to date behaviour.
>
> --
> Viktor.
>
>

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

They don't. But there might be some variable with undesired default value.

# cd /etc/postfix/
lab postfix # grep 'config_directory' main.cf master.cf
lab postfix #


Em sex, 4 de jan de 2019 às 20:11, Viktor Dukhovni
 escreveu:
>
>
>
> > On Jan 4, 2019, at 4:59 PM, Rafael Azevedo  wrote:
> >
> >>> Jan  4 17:41:54 lab postfix/pipe[2027085]: EE5D013E179F:
> >>> to=, relay=post_queue_content_filter, delay=1.1,
> >>> delays=0.04/0/0/1, dsn=4.3.0, status=deferred (temporary failure.
> >>> Command output: postdrop: error: untrusted configuration directory
> >>> name: /etc/postfix/ postdrop: fatal: specify
> >>> "alternate_config_directories = /etc/postfix/" in /etc/postfix/main.cf
> >>> sendmail: warning: command "/usr/sbin/postdrop -r" exited with status
> >>> 1 sendmail: fatal: x...@xx.net(5001): unable to execute
> >>> /usr/sbin/postdrop -r: Success )
> >>
> >> DO NOT set "config_directory = /etc/postfix/" in main.cf.  In fact do
> >> do not set it at all.
> >
> > In fact I didn't.
> > There's no config_directory set in my main.cf file.
>
> The logs don't lie.  Either main.cf has "config_directory = /etc/postfix/"
> with a trailing "/", or Postfix was started via "postfix -c /etc/postfix/ 
> start"
> or you have enabled multiple instances, and the multi-instance configuration
> lists "/etc/postfix/", ... or similar.
>
> By the time "postdrop" is running in your script the "MAIL_CONFIG" environment
> variable is set to "/etc/postfix/" rather than just "/etc/postfix".
>
> --
> Viktor.
>

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

> On Jan 4, 2019, at 5:13 PM, Rafael Azevedo  wrote:
> 
> They don't. But there might be some variable with undesired default value.
> 
> # cd /etc/postfix/
> lab postfix # grep 'config_directory' main.cf master.cf

So the unwanted value was acquired at runtime.

Post the output of:

  # postmulti -l
  # postfix status
  # pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid"; xargs 
-0n1 < /proc/$pid/environ; done

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

# postmulti -l
-   -   y /etc/postfix

# postfix status
postfix: Postfix is running with backwards-compatible default settings
postfix: See http://www.postfix.org/COMPATIBILITY_README.html for details
postfix: To disable backwards compatibility use "postconf
compatibility_level=2" and "postfix reload"
postfix/postfix-script: the Postfix mail system is running: PID: 5158


# pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid";
xargs -0n1 < /proc/$pid/environ; done
PIDPPID COMMAND
12010995158 qmgr -l -t fifo -u -o syslog_name=xx.net
MAIL_CONFIG=/etc/mailservers/xx.net/host.xx.net/
MAIL_LOGTAG=postfix
LANG=C
GENERATION=15302



Em sex, 4 de jan de 2019 às 20:43, Viktor Dukhovni
 escreveu:
>
> > On Jan 4, 2019, at 5:13 PM, Rafael Azevedo  wrote:
> >
> > They don't. But there might be some variable with undesired default value.
> >
> > # cd /etc/postfix/
> > lab postfix # grep 'config_directory' main.cf master.cf
>
> So the unwanted value was acquired at runtime.
>
> Post the output of:
>
>   # postmulti -l
>   # postfix status
>   # pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid"; xargs 
> -0n1 < /proc/$pid/environ; done
>
> --
> Viktor.
>

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

> On Jan 4, 2019, at 6:18 PM, Rafael Azevedo  wrote:
> 
> # postmulti -l
> -   -   y /etc/postfix
> 
> # postfix status
> postfix: Postfix is running with backwards-compatible default settings
> postfix: See http://www.postfix.org/COMPATIBILITY_README.html for details
> postfix: To disable backwards compatibility use "postconf
> compatibility_level=2" and "postfix reload"
> postfix/postfix-script: the Postfix mail system is running: PID: 5158
> 
> 
> # pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid";
> xargs -0n1 < /proc/$pid/environ; done
>PIDPPID COMMAND
> 12010995158 qmgr -l -t fifo -u -o syslog_name=xx.net
> MAIL_CONFIG=/etc/mailservers/xx.net/host.xx.net/
> MAIL_LOGTAG=postfix

So your only "registered" Postfix instance is in /etc/postfix, but
the queue manager is running with:

MAIL_CONFIG=/etc/mailservers/xx.net/host.xx.net/

How is that supposed to work???  And why is "postdrop" logging
problems using "/etc/postfix/" and not "/etc/mailservers/xx.net/host.xx.net/"?

I am afraid you're not providing sufficiently detailed information
about your configuration...  Not much help is possible under these
circumstances.

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

Hi Viktor,

Thanks for your reply.

I've provided all information you asked.

This is a lab server.
It has about 8 IPs and multiple postfix configurations (from older tests).

Although this server has multiple IPs, I'm not running multiple
instances at this time.

The pourpose of this test is to build a postfix server running on
multiple ips, with multiple transports and one self-made anti-spam
system.

So there's a lot of things yet to be accomplished.

In time, would this make any sense:

1. spamfilterunix-   n   n   -   -   pipe
2.user=vmail argv=/home/postfix/app/tools/contentFilter.php
${nexthop} ${size} ${sender} ${recipient}
3./usr/sbin/sendmail -oi -f ${sender} ${recipient}

I'm calling sendmail at line 3 in an attempt to make postfix re-inject
message back into queue.

No success at all.

Any help ..

Thanks!

Em sex, 4 de jan de 2019 às 21:33, Viktor Dukhovni
 escreveu:
>
>
>
> > On Jan 4, 2019, at 6:18 PM, Rafael Azevedo  wrote:
> >
> > # postmulti -l
> > -   -   y /etc/postfix
> >
> > # postfix status
> > postfix: Postfix is running with backwards-compatible default settings
> > postfix: See http://www.postfix.org/COMPATIBILITY_README.html for details
> > postfix: To disable backwards compatibility use "postconf
> > compatibility_level=2" and "postfix reload"
> > postfix/postfix-script: the Postfix mail system is running: PID: 5158
> >
> >
> > # pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid";
> > xargs -0n1 < /proc/$pid/environ; done
> >PIDPPID COMMAND
> > 12010995158 qmgr -l -t fifo -u -o syslog_name=xx.net
> > MAIL_CONFIG=/etc/mailservers/xx.net/host.xx.net/
> > MAIL_LOGTAG=postfix
>
> So your only "registered" Postfix instance is in /etc/postfix, but
> the queue manager is running with:
>
> MAIL_CONFIG=/etc/mailservers/xx.net/host.xx.net/
>
> How is that supposed to work???  And why is "postdrop" logging
> problems using "/etc/postfix/" and not "/etc/mailservers/xx.net/host.xx.net/"?
>
> I am afraid you're not providing sufficiently detailed information
> about your configuration...  Not much help is possible under these
> circumstances.
>
> --
> Viktor.
>

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

On Fri, Jan 04, 2019 at 09:57:53PM -0200, Rafael Azevedo wrote:

> I've provided all information you asked.

Well, but you've provided detailed configuration information.
See http://www.postfix.org/DEBUG_README.html#mail

> This is a lab server.
> It has about 8 IPs and multiple postfix configurations (from older tests).

Well, I don't recall any mention of multiple Postfix configurations
until now.  For "sendmail" to work in a non-default Postfix instance
the associated configuration directory must be listed in the default
instance's main.cf file's "alternate_config_directories" parameter.

> Although this server has multiple IPs, I'm not running multiple
> instances at this time.

Well, you're running a non-default instance, whose configuration
is not /etc/postfix.  It does not matter whether other instances
are running or not, you're in a multi-instance configuration.

> In time, would this make any sense:
> 
> spamfilterunix-   n   n   -   -   pipe
>   user=vmail argv=/home/postfix/app/tools/contentFilter.php ${nexthop} 
> ${size} ${sender} ${recipient}
>   /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Not really, because the sendmail command goes inside the script,
and I'd stay well clear of PHP for processing email and executing
sub-processes, ...

And ${recipient} should always be the last command line variable,
since it expands to multiple recipient arguments.

> No success at all.

You're stumbling around in the dark trying random things that can't
work.  Find a native English speaker who can help you understand
FILTER_README, and get it working in a single-instance Postfix
environment first.

Consider buying the Postfix Book by Patrick and Ralf, and read the
chapter(s) on content filters.

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

> Well, I don't recall any mention of multiple Postfix configurations
> until now.  For "sendmail" to work in a non-default Postfix instance
> the associated configuration directory must be listed in the default
> instance's main.cf file's "alternate_config_directories" parameter.

Ok, got it back to default config.
# postmulti -l
-   -   y /etc/postfix

# pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid";
xargs -0n1 < /proc/$pid/environ; done
PIDPPID COMMAND
2028229 2026262 qmgr -l -t unix -u
MAIL_CONFIG=/etc/postfix/
MAIL_LOGTAG=postfix
LANG=C
GENERATION=724

# postfix -c /etc/postfix/ status
postfix: Postfix is running with backwards-compatible default settings
postfix: See http://www.postfix.org/COMPATIBILITY_README.html for details
postfix: To disable backwards compatibility use "postconf
compatibility_level=2" and "postfix reload"
/usr/sbin/postconf: warning: /etc/postfix/master.cf: undefined
parameter: mua_sender_restrictions
/usr/sbin/postconf: warning: /etc/postfix/master.cf: undefined
parameter: mua_client_restrictions
/usr/sbin/postconf: warning: /etc/postfix/master.cf: undefined
parameter: mua_helo_restrictions
postfix/postfix-script: the Postfix mail system is running: PID: 2026262

Same behavior..

> Not really, because the sendmail command goes inside the script,
> and I'd stay well clear of PHP for processing email and executing
> sub-processes, ...

what do you mean by inside the script?

> And ${recipient} should always be the last command line variable,
> since it expands to multiple recipient arguments.

thanks, my mistake

> You're stumbling around in the dark trying random things that can't
> work.  Find a native English speaker who can help you understand
> FILTER_README, and get it working in a single-instance Postfix
> environment first.

yes, at this time.

> Consider buying the Postfix Book by Patrick and Ralf, and read the
> chapter(s) on content filters.

This one?
https://books.google.com.br/books?id=uq1Et7KmsGwC&pg=PA22&dq=postfix&hl=pt-BR&sa=X&ved=0ahUKEwjC9evHr9XfAhXFgZAKHb2zBEIQ6AEIPzAD#v=onepage&q=postfix&f=false

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

On Fri, Jan 04, 2019 at 10:22:28PM -0200, Rafael Azevedo wrote:

> Ok, got it back to default config.
> # postmulti -l
> -   -   y /etc/postfix
> 
> # pgrep -x qmgr | while read pid; do ps -o pid,ppid,args -p "$pid";
> xargs -0n1 < /proc/$pid/environ; done
> PIDPPID COMMAND
> 2028229 2026262 qmgr -l -t unix -u
> MAIL_CONFIG=/etc/postfix/
> MAIL_LOGTAG=postfix

No you did not.  By default "MAIL_CONFIG=/etc/postfix" with no trailing "/".
Perhaps you're starting Postfix via:

postfix -c /etc/postfix/ start

don't do that.

-- 
Viktor.

Re: Content filter - reijnect message back into queue

[Rafael Azevedo]

> No you did not.  By default "MAIL_CONFIG=/etc/postfix" with no trailing "/".
> Perhaps you're starting Postfix via:
>
> postfix -c /etc/postfix/ start
this is how we start postfix intances since this server has multiple
testing configurations.


> don't do that.
why?

and why shall I keep away from php for this content filter script? I
think its fast (isn't it?)

I want to easily be able to compare last message with a database and
try to match if the sending message is bulk or not and drop it if so.

thanks a lot.

Re: Content filter - reijnect message back into queue

[Viktor Dukhovni]

On Fri, Jan 04, 2019 at 10:40:21PM -0200, Rafael Azevedo wrote:

> > postfix -c /etc/postfix/ start
>
> this is how we start postfix intances since this server has multiple
> testing configurations.

Because the correct command is just "postfix start".  And you want
multiple instances read MULTI_INSTANCE_README, and configure them
properly.

> > don't do that.
> why?

Because it causes the problem you're reporting, by supplying a
subtly non-default configuration directory.

> and why shall I keep away from php for this content filter script? I
> think its fast (isn't it?)

Because it is too easy to create security issues in PHP scripts.

> I want to easily be able to compare last message with a database and
> try to match if the sending message is bulk or not and drop it if so.

Try Python or Perl, and use a command-execution interface that
bypasses the shell, and directly calls execve(2) or execvp(3) with
an array of arguments.  Make sure to separate the first recipient
address from the last option with "--", and make sure to pass "-f",
"sender-address" as two separate arguments.

-- 
Viktor.

How to configure an infinite-retry for relay

[Paul Goyette]

I have a situation where my primary/final MX server will be down for
an indefinite period of time, possibly up to a week.  During that time
I would like to have the secondary MX server to keep every message
queued, and keep on retrying, without ever "timing out" and without
sending any "undeliverable" notifications to the sender.

I've tried to find relevant configuration variables in the docs, but
haven't found anything.

Putting up a new "final" MX server really isn't an option, as I and
the server will both be in the middle of a trans-oceanic relocation.





+--+--++
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:  |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee dot com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd dot org |
+--+--++