Hi Viktor,
What sort of "local filter" are you talking about?
local filter is smtpsplit transport
Why? Postfix does not by default do that, unless you configure self-delivery.
What is the purpose of the internal delivery?
The purpose is to have at some conditions email per recipient
What new emails? Your problem description is much too fuzzy.
New email is emails that get created in queue after split for each rcpt
from origin email, origin email is email that has many rcpts. When there
no OOM origin email get properly mark each spitted rcpt as "done" and no
duplicates appear.
What and why is "smtpsplit"?
Name of smtp transport used as filter to pass emails per 1 rcpt to
special smtpd on localhost
You may be processing each message through your milters twice.
Yep, and this is intended behavior. Basically milter is who instruct to
add Split header and instead of doing part of logic which impossible or
split-brain-complicated for multi recipients emails.
It's common thing when you have in milter stuff which can be properly
proccessed only if there 1 rcpt. Like one rcpt want accept email and
second - doesn't.
With server systems routinely shipping with 256GB of RAM or more these days,
100MB seems rather undersized. Don't do that.
Well I don't have such RAM limits in production, it was reproduced on
test setup, but the point that production setup still have a limits,
like 4gb per instance and if OOM can result in such situation on test
stand, it can get same results in production too, that's why I worried.
I propose set 100MB not because I plan to run Postifx with such low
amount of RAM, but to provide people here ability to reproduce issue I
speak about and maybe find a way to improve code to avoid duplicated
emails at OOM.
Regards,
Dmytro Alieksieiev
On 04/04/2025 03:32, Viktor Dukhovni via Postfix-users wrote:
On Thu, Apr 03, 2025 at 10:17:22PM +0200, Dmytro Alieksieiev via Postfix-users
wrote:
I faced issue with splitting emails via local filter and
destination_recipient_limit = 1 under low memory condition which
result in duplicated emails in queue.
Not duplicated, simply retained, if/when information that a recipient is
delivered did get recorded in the queue file. What sort of "local
filter" are you talking about?
1. Postfix get email with multiple rcpts
2. taking couple of rcpts (depending on settings, 3 by default) and creates
new email in queue per each recipient
Why? Postfix does not by default do that, unless you configure
self-delivery. What is the purpose of the internal delivery?
3. reach OOM
That's your actual problem. Perhaps your filter uses unreasonable
memory, tune the filter and its concurrency, if that's a factor in
its memory requirements.
4. new emails stay in queue, but original email not get rcpt marked as
"done", as result on next queue rerun they will be spitted again resulting
in duplicated messages
What new emails? Your problem description is much too fuzzy.
Can be reproduced in docker with mem limit f.e. 100mb by send one email with
50 rcpts with next configuration:
With server systems routinely shipping with 256GB of RAM or more these
days, 100MB seems rather undersized. Don't do that.
1. Extra main.cf:
smtpsplit_destination_recipient_limit = 1
What and why is "smtpsplit"?
2. Extra master.cf:
smtpsplit unix - - n - - smtp
-o syslog_name=postfix-split
-o smtp_send_xforward_command=yes
-o disable_mime_output_conversion=yes
-o smtp_generic_maps=
-o smtp_header_checks=
What is the intent here? You should probably set a lower process limit
on this service.
127.0.0.1:5025 inet n - n - - smtpd
-o syslog_name=postfix-after-split
-o content_filter=
-o cleanup_service_name=cleanupsplit
-o receive_override_options=no_unknown_recipient_checks
-o smtpd_helo_restrictions=
-o smtpd_client_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_relay_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_tls_security_level=none
-o smtpd_data_restrictions=
You may be processing each message through your milters twice.
_______________________________________________
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
