Hi Wietse,
Thanks for the pipemap:{pcre, socketmap} suggestion. I tested both
variants with postmap -q against real Postfix (3.9.11 and 3.11.5), which
also covers the UTF-8 concern.
Two facts confirmed:
First, pcre: patterns count bytes, not characters. A 299-character /
300-byte key (298 × 'a' followed by a two-byte é) trips if !/(.{300})/,
confirming .{300} matches on byte length.
Second, Postfix validates the rewrite result as UTF-8 before returning
it. This is what separates the two variants:
The negative pattern
if !/(.{300})/
/(.+)/ $1
ENDIF
never rewrites the key. It returns the whole, unmodified string when it
is under the limit, and no result for anything at or above 300 bytes, at
which point pipemap short-circuits and the socketmap is never queried.
Because the key is never truncated, there is no encoding hazard
regardless of where a multibyte character sits. Side note: I changed
(.*) to (.+) as empty string result is not allowed in maps.
The positive pattern /^(.{1,300})/ $1 truncates at a byte boundary. When
that boundary falls inside a multibyte character, Postfix rejects the
malformed result with:
warning: ... non-UTF-8 value "...": malformed UTF-8 or invalid codepoint
fatal: table pcre:...: query error
A query error is a temporary failure, so the lookup defers rather than
returning a clean result.
To quantify it, I placed a four-byte character (😅, F0 9F 98 85) across
the 300-byte boundary at each alignment: three of the four offsets
produce a query error, and only the alignment where the character ends
exactly at byte 300 survives.
The failure is therefore data-dependent and intermittent.
So the negative pattern is the right fit for shielding an uncooperative
socketmap from over-long input, without touching Postfix internals and
without any UTF-8 truncation risk. Thanks again for the pointer.
As a longer-term direction, it would be valuable to be able to return an
action — a 5.x.x REJECT — based on a header address before cleanup
commits the message, rather than only accept-or-defer. That is not
specific to recipient_canonical_maps; it is the general gap that today a
malformed header address gives an operator no clean way to refuse the
message at that stage.
One point worth flagging for /anyone/ running a split scheme like mine:
envelope_recipient and header_recipient share the same
recipient_canonical_maps, so this length guard applies to both
indiscriminately. That makes it important to also enforce a
recipient-length limit at the incoming relay (the RCPT check), and to
keep the two limits in sync with a deliberate window gap — the incoming
limit set safely below the guard's threshold. Without that gap, an
over-long but otherwise legitimate envelope recipient would trip the
guard and SRS would simply be skipped for it, instead of the guard only
ever catching the oversized header-derived key it is meant for.
Regards,
Dmytro Alieksieiev
DevOps Engineer
On 15/07/2026 23:36, Wietse Venema via Postfix-users wrote:
Dmytro Alieksieiev via Postfix-users:
Hi Wietse,
Thanks - and glad we agree on the PERM/451 point; that alone would take
most of the pain out of this.
That is easier said than done.
Postfix table lookups currenly have three possible results:
- DICT_ERR_NONE: the requested item was found, or it does not exist.
- DICT_ERR_RETRY: typically, timeout
- DICT_ERR_CONFIG: configuration error
The difference between DICT_ERR_RETRY and DICT_ERR_CONFIG is in the
error message; both result in the same program behavior, because
configuration errors can be corrected.
The Postfix mapping is:
OK, NOTFOUND -> DICT_ERR_NONE
TEMP -> DICT_ERR_RETRY
PERM -> DICT_ERR_CONFIG
Changing the maning of DICT_ERR_CONFIG into a "reject" would be a
compatibility break, and adding a new "reject" status would require
lots of changes everywhere Postfix looks up information.
Until now there has een no use case for dictionaries to
return an error that says "reject".
On header_checks: I don't think it can cover this one. header_checks
match a header as a single logical blob, not per-address, so I can't
express "reject/shorten the one over-long address token" ? and
rewriting/truncating text inside a folded, quote-swallowed header is
fragile (the very thing that makes it dangerous, the unterminated quote,
is what makes it hard to match a stable pattern against).
Inded, this is an available option, now.
Here's another option:
xxx_maps = pipemap:{prce:/path/to/file,socketmap:{your-stuff}}
In the first example, /path/to/file contains a negative pattern to
select with < 300 bytes. Longer strings will produce a "not found"
result without using the socketmap:
if !/(.{300})/
/(.*)/ $1
ENDIF
In the second example, /path/to/file uses a positive pattern to
to expose the socketmap only to the first 300 bytes of a string.
/^(.{1,300}) $1
No idea how this handles UTF-8 non-ASCII.
This may be a reasonable option for making Postfix work with
an uncooperative socketmap.
Wietse
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
